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1.  SCOPE. 

a.  This  Test  Operations  Procedure  (TOP)  describes  a  systematic  approach  to  “safety, 
performance,  and  reliability”  testing  of  dismounted  handheld  detection  systems  used  for 
landmines,  improvised  explosive  devices  (IED)  and  their  components  (e.g.,  pressure  plates, 
command  wire,  triggering  devices),  unexploded  ordnance,  and  buried  ammunition  caches.  The 
objective  of  the  procedures  outlined  in  this  TOP  is  to  provide  methods  of  assessing  the  technical 
performance  effectiveness,  engineering,  safety,  and  technical  characteristics  of  these  dismounted, 
handheld  detection  systems  under  test  (SUT).  This  document  will  point  to  other  TOPs,  Military 
Standards  (MIL-STD),  and  if  necessary.  International  Test  Operations  Procedures  (ITOP)  and 
North  Atlantic  Treaty  Organization  (NATO)  Standardization  Agreement  (STANAG),  for  more 
detailed  explanation  of  specific  test  activities. 

b.  The  TOP  also  provides  the  basis  for  comparing  present  techniques  and  instrumentation 
and  for  improving  and  optimizing  existing  technologies  (in  terms  of  sensor  efficiency  and 
effectiveness,  reduction  of  false  alann  rate,  time  required  for  operations,  etc.).  However,  this 
TOP  is  seen  as  a  crucial  aspect  for  the  development  of  new  technologies,  and  it  is  recognized  that 
this  TOP  will  positively  contribute  to  increase  the  credibility  when  a  new  prototype  is  introduced. 

1.1  Purpose. 

The  purpose  of  this  TOP  is  to  provide  the  tester  standardized  testing  methodologies  and 
procedures  to  assess  handheld  detection  systems  in  order  to  test  the  overall  safety,  performance, 
and  reliability  of  the  system.  It  describes  activities  necessary  to  ensure  safety  is  designed  into  the 
system  under  test,  and  to  verify  detection  performance  meets  system  requirements  (to  detect 
landmines,  IEDs  and  their  components,  as  well  as  ammunition  cache  and  unexploded  ordnance). 

1.2  Applicability. 

This  document  is  appropriate  for  hand-held  detection  systems  that  are  carried  in  a  dismounted 
configuration.  Today,  detection  systems  are  being  integrated  with  ground  penetrating  radar 
sensors,  and  cannot  strictly  be  focused  on  metal  detection  applications.  Aspects  of  testing 
detection  systems  with  single  or  dual  sensing  capability  will  be  introduced  and  addressed.  It  will 
be  up  to  the  tester  to  detennine  which  subtests  are  applicable  to  the  system  under  test,  and  which 
tests  are  required  to  meet  system  assessment  requirements. 

1.3  Limitations. 

a.  Various  levels  of  engineering  development  and  production  models  were  considered  in 
the  development  of  this  document.  Again,  it  will  be  up  to  the  assessment  team  to  determine 
which  level  of  testing  is  required. 

b.  This  document  is  applicable  to  testing  of  multiple  handheld  detection  systems  working 
in  the  vicinity  of  each  other;  however,  testing  of  multiple  detection  systems  working  in 
cooperation  is  not  addressed  in  this  document. 
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1.4  Introduction  to  Dismounted  Hand-Held  Detection  Systems  and  Modem  Warfare  Threats. 

a.  With  the  onset  of  IEDs,  detectors  have  had  to  expand  target  sets  from  the  typical 
conventional  buried  landmine  to  IEDs  (and  their  components),  command  fire  wires,  hidden 
ammunition  cache,  and  unexploded  ordnance.  Having  dual  sensor  detection  systems  allows  the 
user  more  ability  (capability)  to  locate  the  threat.  However,  not  all  applications  require  dual 
sensor  requirements.  Common  metal  detection  systems  are  still  in  high  demand.  They  are 
typically  lighter  weight,  easier  to  operate,  and  cost  less  than  their  dual  sensor  counterpart. 

Bottom  line,  it  is  up  to  the  application  and  operator  to  understand  and  know  the  detector  to  fully 
take  advantage  of  its  capabilities. 

b.  The  known  or  suspected  use  of  landmines  or  IEDs  by  an  enemy  threat  in  ground 
warfare  necessitates  some  action  to  neutralize  or  reduce  the  threat  when  new  terrain  is  to  be 
traveled  or  occupied  by  friendly  forces.  Anti-personnel  and  anti-vehicle  explosive  devices  are 
produced  in  a  variety  of  sizes,  shapes,  and  operational  design,  and  constantly  changing  in  overall 
configuration.  These  threats  are  constructed  of  metallic  and  non-metallic  materiel.  They  are 
usually  buried,  or  otherwise  concealed  in  roads,  trails,  paths,  concrete  curbs,  culverts,  etc.,  in 
areas  likely  to  be  traversed  by  friendly  troops,  civilians,  or  vehicles. 

c.  Technology  has  simplified  the  design  of  detection  systems  causing  a  reduction  in  size 
and  weight,  allowing  faster  processing  speeds,  and  more  enhanced  algorithms  in  the  end  goal  to 
increase  detection  performance,  and  reduced  false  alarms.  But  once  again,  it  is  up  to  the  operator 
to  determine  target  versus  false  alarms  caused  by  ground  and  natural  clutter.  The  operator  must 
be  fully  trained  on  the  system  being  tested  and  know  and  understand  the  various  sensitivity 
levels  of  the  detector  functionality. 

2.  FACILITIES  AND  INSTRUMENTATION 

2.1  Test  Facilities. 

2.1.1  Sensor  Experimentation  Facility. 

a.  This  type  of  test  facility  is  designed  to  provide  for  performance  characterization  under 
highly  controlled  conditions  of  soil,  soil  moisture  content,  light  conditions,  and/or  temperature 
conditions.  Soils  are  normally  homogeneous,  pristine  soils  (sand,  clay,  loam,  etc.). 

Homogenous  soil  samples,  chemically  and  physically  well  characterized,  typically  have  well 
defined  electromagnetic  (EM)  properties  (electrical  permittivity  and  conductivity,  magnetic 
permeability,  etc.). 

b.  This  type  of  facility  is  normally  used  for  prototype  concepts.  NOTE:  Sensor 
Verification  trials  or  system  performance  trails  are  discussed  later  in  this  document.  Within  this 
environmentally  controlled  facility,  it  is  of  most  utility  when  applied  to  multiple  candidates  to 
characterize  limits  of  (or  to  bench  mark)  performance.  The  facility  may  have  integrated  robotics 
to  move  detector  heads  at  controlled  velocities,  directions,  tilt,  and  height  to  evaluate  the  effects 
of  these  conditions  independent  of  human  operator,  especially  for  each  set  of  environmental 
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conditions.  Cartesian  gantries  (x,y,z)  may  be  used  to  provide  a  controlled  method  of  maintaining 
these  vectors  and  scalar  parameters. 

c.  Care  should  be  taken  to  ensure  that  the  sizing  and  construction  of  this  type  of  facility 
does  not  compromise  test  results.  That  is,  certain  types  of  sensors  may  be  extremely  sensitive  to 
either  facility  (soil  bin),  size  (height,  width,  and  depth),  or  construction  material.  Depending 
upon  the  type  of  sensors,  edge  effects  may  occur  at  the  sides  of  the  soil  bin,  limiting  the  useable 
scan  area.  Additionally,  for  certain  type  of  sensors,  inadequate  depths  may  induce  unwanted  or 
spurious  reflections.  If  devices  such  as  Cartesian  gantries  are  used,  nonmetallic  or  other  suitable 
materials  should  be  used  for  the  carriage,  to  ensure  results  are  not  compromised. 

2.1.2  Calibration/Data  Collection  Detection  Test  Ranges. 

As  a  test  control  and  an  aid  in  diagnostics  of  detector  technology  development  and  qualification, 
it  is  recommended  that  special  calibration  and  data  collection  areas/lanes  containing  buried 
targets  be  established  and  dedicated  for  detector  calibration,  training,  engineering  development, 
and/or  verifying  functionality  of  the  detection  equipment.  During  hardware  and  algorithm 
development,  data  collection  activities  allow  maturity  growth  of  the  detection  system  over  areas 
that  can  be  repeated  over  and  over  to  allow  perfonnance  comparison.  Typically,  the  buried 
target  locations  are  known  to  the  detector  operators  (e.g.,  marks  on  ground,  target  map,  grid, 
etc.).  These  calibration/data  collection  areas  allow  informal  assessment  without  comprising  the 
formal  evaluation  test  area  where  targets  are  unknown  to  the  operators  (e.g.,  “blind”  test 
areas/lanes).  (Note:  the  blind  test  areas/lanes  should  have  restricted  access  until  a  full 
performance  evaluation  is  required.)  The  targets  buried  in  these  lanes  would  be  similar  to  targets 
buried  in  the  blind  evaluation  test  areas/lanes. 

2.1.3  Performance  Detection  Test  Range. 

The  Performance  Detection  Test  Range  should  be  representative  of  the  environmental  categories 
and  conditions  that  the  detection  equipment  is  intended  to  be  used  in.  Two  basic  approaches  are 
suggested  for  selection  of  detection  test  areas:  a  global  classification  approach  and  a  simplified 
approach.  In  the  global  approach,  a  list  of  typical  soils  found  in  areas  of  hostile  territories  is 
developed  and  simulated  conditions  should  be  established.  These  soils  should  be  prioritized  by 
independent  experts  in  terms  of  probability  of  encounter  or  percentage  of  occurrence.  The  soils 
should  then  be  grouped  by  similar  physical  properties  using  a  soil  classification  method.  Once 
accomplished,  detection  test  areas  should  be  selected  that  represent  the  highest  probability  of 
encounter  (nominal)  with  possible  off-nominal  conditions  selected,  if  time  pennits.  In  the 
simplified  approach,  a  narrow  set  of  soils  (e.g.,  sand,  loam,  and  clay)  should  be  selected  that 
represent  the  expected  boundary  conditions  of  perfonnance  for  a  specific  sensor  from  worst  to 
best,  or  the  use  of  existing  soil  configurations,  but  in  specific  environmental  considerations  (e.g., 
arctic  frozen  tundra,  tropics  wet  rainforests,  or  hot  dry  desert,  sandy  beaches,  loamy  farmland, 
etc.).  However,  using  specific  environmental  areas  are  considered  advanced  testing  because  it 
introduces  challenges  associated  with  the  soil  conditions  containing  more  uncontrollable 
variables  (e.g.,  defrosted  and  frozen  ground,  water/rain  saturated  soils,  vegetation  and  rodent 
cavities,  etc.).  Appendix  C  shows  an  example  of  the  profiling  process  for  soils.  These  “Blind” 
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lanes/areas  have  buried  targets  that  are  unknown  to  the  operator  or  detection  team.  The  ground 
truth  is  maintained  as  sensitive  and  used  for  scoring  the  detector  perfonnance  of  detection. 

2.2  Instrumentation. 

a.  Instrumentation  requirements  must  be  considered  when  testing  a  dismounted  hand¬ 
held  detection  system.  Handheld  detection  systems  may  have  data  logger  or  computer  interfaces 
to  monitor  or  capture  waveform  or  digital  data  responses  over  targets,  while  the  detector  is 
scanning  the  test  area.  This  information  also  can  be  used  to  validate  system  models  and  allow 
for  test  repeatability  that  would  not  normally  be  available  without  this  infonnation.  Data 
instrumentation  packages  used  must  be  able  to  pull  information  from  the  detection  system 
without  causing  a  power  drain  or  create  an  additional  heat  source  which  may  affect  reliability  or 
performance.  Typically,  this  instrumentation  integration  must  be  done  by  the  manufacturer  to 
minimize  any  interface  problems  to  permit  this  parallel  data  capture. 

b.  Time  and  Position  Tracking  Instrumentation  may  be  used  on  top  of  the  detector  head 
and  on  the  operator  to  detennine  the  path  of  sensor/operator  over  target  area.  The  information 
recorded  permits  analysis  of  how  the  operator  works  the  sensor  over  target  area  to  validate  the 
head  passes  over  potential  targets.  If  an  operator  is  not  scanning  over  test  areas,  additional 
training  of  scanning  techniques  must  be  implemented. 

c.  All  instrumentation  used  should  be  documented  in  the  test  plan  and  final  report,  and 
the  description  of  the  instrumentation  should  include  the  accuracy  or  permissible  measurement 
uncertainty  of  the  instrumentation.  Photographs  of  the  instrumentation  configuration  and  usage 
should  be  included  in  the  final  report. 

2.3  Standardized  Detection  Targets. 

Targets  should  consist  of  a  common  target  set  to  allow  standardization  (to  allow  target 
duplication)  between  test  targets  within  the  test  range  and  other  test  sites.  Main  charges, 
boosters,  and  detonator  metallic  material  should  be  included  in  each  target  to  ensure  realistic 
detection  characteristics.  If  components  are  not  included,  or  substituted  with  another  material, 
ground  penetrating  radar  (GPR)  signals  may  not  reflect  accurate  responses.  Examples  of 
detection  targets  in  the  test  lanes/areas  could  include,  but  are  not  limited  to,  the  following: 

a.  Anti-tank  landmines,  metallic  and  low  metallic:  examples  such  as  Ml 5,  TM62-P3, 
VS-1.6,  VS-2.2,  and  M19. 


b.  Anti-personnel  landmines,  metallic  and  low  metallic:  examples  such  as  M14,  TS50, 
VS50,  PMN,  PMN-2,  and  PMA-3. 

c.  IED  targets:  examples  such  as  plastic  jug  (non-metallic),  sand  filled  pressure  cooker 
(metallic),  155-millimeter  (mm)  projectiles,  and  120-mm  mortars. 
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d.  IED  Triggering  Devices:  examples  such  as  low  metallic  pressure  plates  (carbon  core, 
tire),  non-metallic  pressure  plates  (e.g.,  wood  double  plunger),  dual  conductor  lamp  cord,  and 
single  conductor  enamel  coated  copper  wire. 

e.  Clutter:  natural  ground  clutter  pre-existing  in  test  areas/lanes  (rocks,  wood,  roots, 
wash  beds,  and  rodent  holes.) 

f.  Man-Made  Clutter:  examples,  such  as,  nails,  nuts,  bolts,  plastic  bottles,  aluminum 
cans,  and  rope. 

2.4  Test  Controls. 

a.  An  existing  detection  test  facility  should  be  used  when  possible.  It  is  necessary  to 
consider  all  types  of  climatic  and  environmental  conditions  in  a  great  variety  of  soil  types,  so 
several  test  ranges  may  be  considered  for  a  thorough  assessment  of  a  detection  system.  These 
test  areas  should  be  monitored  for  air  and  ground  temperatures,  and  soil  moisture  content.  The 
test  areas/lanes  should  have  soil  characteristics/profiles  mapped  and  documented  and  available 
for  detector  algorithm  development.  The  tester  should  consider  the  effects  of  operating  on 
improved  ground/road  conditions  and  undisturbed  terrain,  with  the  possibility  to  operate  in  and 
around  natural  and  man-made  obstacles  (e.g.,  chain  link  or  barbed  wire  fences,  brick  or  adobe 
walls,  wash  beds  (wadis),  trees  or  shrubbery,  etc.).  Other  natural  and  man-made  conditions  may 
consist  of  snow,  mud,  saturated  wet  soils,  rocky  grounds,  and  asphalt  versus  gravel  versus  dirt 
roadways. 

b.  The  tester  should  also  consider  the  effects  of  high  and  low  temperature  extremes, 
humidity,  snow,  freezing  rain,  sand,  dust,  vibration,  and  electromagnetic  interference.  These  are 
subtests  typically  performed  in  climatic  chambers  or  specialized  test  facilities  to  assess 
durability,  reliability,  and  electromagnetic  compatibility/interoperability  of  the  system  under  test. 
Sometimes  during,  but  usually  after  being  subjected  to  a  test,  functional  checkouts  are  perfonned 
to  assess  system  operations  and  performance. 

c.  Handheld  detection  systems  should  be  tested  in  facilities  where  there  are  no  other 
experimental  systems  under  test  (e.g.,  ground  penetrating  or  forward  looking  radar  system, 
telemetry  data  transfer  system,  remote  control  robotics  operations,  etc.). 

3.  REQUIRED  TEST  CONDITIONS. 

3.1  Test  Item  Configuration. 

In  general,  the  item  should  be  tested  at  system  level,  and  as  the  latest  model  version  or 
production  ready,  unless  it  can  be  shown  that  system  integrity  does  not  contribute  to  the  specific 
results. 


a.  Specially  instrumented  subsystems  may  be  included  to  provide  a  real-time  record  of 
detection  trials.  As  an  example,  time  and  position  sensors  can  be  attached  or  embedded  by 
design  as  special  test  instrumentation  to  provide  integrated  reporting  of  detection  alerts  or  marks 
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as  a  function  of  time  and  geodetic  position  (Global  Positioning  System  (GPS)  coordinates). 
Instrumented  systems  or  subsystems  should  be  used  to  enhance  quality  assurance,  in-test 
inspections,  failure  analysis  or  diagnostics,  and  data  acquisition,  where  it  can  be  shown  that 
results  are  not  compromised  or  that  results  are  better  quantified  (area  of  coverage).  Care  should 
be  taken  to  ensure  that  integral  instrumentation  is  sufficiently  rugged  to  withstand  and  operate 
following  natural  and  induced  environments.  Additionally,  care  should  be  taken  to  ensure  that 
the  presence  of  instrumentation  does  not  interfere  with  the  man-machine  interface  or  bias  human 
factors  appraisals. 

b.  The  Program  Manager  and/or  Materiel  Developer  should  identify  and  provide  the 
detector  systems  to  the  test  center  in  advance  to  allow  a  proper  initial  inspection  and  training 
period  prior  to  the  start  of  record  detection  tests.  Also  included  in  the  delivery,  should  be  all  the 
necessary  power  supplies  (batteries)  and  chargers,  unless  prior  coordination  has  been  perfonned 
to  allow  the  test  center  to  procure  or  obtain  the  necessary  support  items  in  advance. 

c.  Technical/Operator  manuals  (TM/OM)  must  be  provided  with  the  items  under  test. 

For  either  published  or  draft  manuals,  the  test  center  must  review  the  material  for  accuracy  of 
procedures  and  to  ensure  the  safety  and  warnings  are  adequate  to  inform  the  operator,  as  well  as, 
to  protect  the  operator  and  test  personnel  from  possible  safety  issues. 

3.2  Test  Planning. 

a.  Test  planning  should  be  initiated  early  in  the  acquisition  process  in  order  to  yield  the 
most  cost  effective  approach  to  decision  risk  mitigation.  Material  developer  should  consider 
software,  component,  and  subsystem  level  testing  throughout  the  early  development  of  the 
detection  system.  Each  version  must  demonstrate  effective  support/operation  of  the  system.  The 
level  of  testing  should  be  carefully  considered  (e.g.,  data  collection  versus  blind  evaluated 
testing).  At  some  point,  the  design  and  algorithm  must  be  set  to  progress  to  system  level 
performance  assessment  testing  in  blind  test  areas/lanes. 

b.  Testing  should  be  conducted  to  demonstrate  that  the  logistics  supportability  is 
adequate.  Issues  in  the  areas  of  operator  maintenance,  transportation  and  handling  protection, 
storage,  and  technical/operators  manuals  must  be  addressed  along  with  availability,  accuracy, 
and  comprehensibility. 

c.  Test  plan  document  should  address  the  following: 

(1)  Purpose/Scope  of  testing. 

(2)  Test  objectives. 

(3)  Test  criteria  (if  being  evaluated). 

(4)  Test  method/procedures. 

(5)  Data  required. 
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(6)  Data  analysis  techniques  for  detection  scoring. 

NOTE:  Scoring  results  and  data  elements  should  be  addressed  during  all  aspects  of  testing: 
Probability  of  Detection,  false  alarm  rate,  and  advance  rate,  as  well  as,  reliability,  availability, 
and  maintainability  (RAM)  assessment  factors,  Human  Factors  Engineering  considerations,  and 
safety  overview  monitoring.  The  classification  levels  of  the  raw,  processed,  and  scored  data 
(e.g.,  performance  of  detection),  should  be  identified  and  recognized  by  all  parties  - 
Item/Program  Manager,  testers,  and  security  personnel  -  and  properly  documented  prior  to  the 
start  of  any  data  acquisition  testing  phases.  This  will  prevent  possible  misinterpretation  of  data 
by  new  personnel  not  familiar  with  scoring  practices  which  may  cause  security  concerns  during 
post  test  assessments. 

d.  Criteria  for  each  subtest  topic  will  be  defined  based  on  applicable  contractual 
specifications  and  military  requirements  documents.  Where  the  user  criteria  are  not  clearly 
specified,  the  tester  must  develop  criteria  from  the  relevant  ITOP,  TOP,  MIL-STD,  STANAG, 
contractual  requirements,  military  specifications,  or  customer  test  plans.  Development  of  criteria 
should  consider  the  logistics  concept,  deployment  Concept  of  Operations  (ConOps),  intended 
application,  novel  technologies,  and  threat  environment,  as  appropriate.  Failures  and  successes 
should  be  defined  to  provide  for  a  clear  understanding  of  relevant  conclusions  for  certain  subtest 
data  topics  (i.e.,  reliability,  performance,  climatic  suitability,  electromagnetic  interference,  and 
safety). 


e.  The  testers  must  establish  data  requirements  and  analytical  techniques  necessary  to 
address  objectives,  criteria,  and  performance,  and  to  identify  which  data  requirements  will  be 
satisfied  through  physical  test.  Test  personnel  should  design  data  and  check  sheets  for  each 
subtest  data  topic. 

f.  Appropriate  test  procedures  for  each  subtest  topic  relative  to  objectives,  criteria,  and 
data  requirements  should  be  selected.  These  proposed  procedures  should  be  discussed  with 
customer,  operators,  and  subject  matter  experts  to  ensure  adequate  or  correct  techniques  are  used. 
Detailed  procedures  then  can  be  drafted  and  outlined  in  the  test  plan  and/or  Standard  Operation 
Procedures  (SOP). 

g.  As  a  tool  to  audit,  change,  or  summarize  the  test  program,  it  is  desirable  to  layout  all 
events  and  data  requirements  in  matrix  format.  Multiple  matrices,  each  of  which  may  be  linked, 
may  be  required  and  would  best  be  displayed  in  a  flow  chart  matrix.  Identify  any  unique 
resources  necessary  to  conduct  subtests. 

h.  The  item  developer  should  identify  how  modeling  and  simulation  will  be  used  to 
answer  test  objectives  which  should  be  outlined  within  the  detailed  test  plan. 

i.  Example  of  the  test  team  man-power  requirements  for  test  execution  and  supporting 
personnel  are  outlined  below: 
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(1)  Test  Director  (a.k.a.,  tester).  Generates  the  test  plan,  prepares  test  lanes, 
coordinates  the  support  personnel  requirements,  assists  with  the  development  of  the  test  matrix, 
executes  the  test,  manages  the  matrix  for  execution,  and  writes  the  test  report  and  any  other 
follow-on  supporting  reports  or  safety  documentation.  The  tester  may  also  serve  at  the  Range 
Safety  Officer. 

(2)  Data  Analyst.  Develops  the  matrix,  receives  the  data,  controls  the  ground  truth, 
and  manages,  analyzes,  and  scores  detection  data.  The  matrix  must  be  based  on  number  of  target 
type  encounters,  per  each  burial  depth,  per  each  soil  configuration.  The  quantity  of  encounters 
example  for  a  statistical  confidence  achievement,  based  on  the  number  of  encounters,  are  shown 
in  Table  1. 


TABLE  1.  SAMPLE  OF  THE  STATISTICAL  CONFIDENCE  EQUIVALENCY  BY  THE 

NUMBER  OF  TARGET  ENCOUNTERS. 


PERCENT  STATISTICAL 
CONFIDENCE 

NUMBER  OF  ENCOUNTERS  AT  THE  SAME 
PARAMETERS3  FOR  EACH  TARGET  TYPE 

90% 

30 

87% 

27 

83  % 

21 

80% 

18 

aTest  Parameters:  depth,  soil  type,  operator 

(3)  Data  Collectors.  Monitor  and  track  perfonnance  and  operational  parameters  of 
the  detector  system  (e.g.,  times,  operators,  SUT),  assist  in  the  placement  of  alarm  markers  along 
detection  routes,  and  generate  log  books  and  incident  reports.  The  Data  Collector  may  also 
record  operator  observations  and  comments  while  scanning  lanes  with  detectors  under  test.  This 
also  includes  monitoring  the  maintenance  personnel  that  perform  repairs  or  software  updates  to 
record  into  log  books. 

(4)  System  Operators.  Operate  the  detection  systems  under  test,  scan  the  target  areas, 
indicate  whether  the  target  is  a  valid  target  or  possible  false  alarm,  identify  where  alann  markers 
are  placed  on  the  lane  to  detail  the  location  of  a  target,  provide  feedback  on  the  operation  of  the 
detector  systems,  and  complete  human  factors  surveys. 

(5)  Geodetic  Surveyors:  Operate  the  GPS  survey  equipment,  and  survey  alarm 
marker  positions  providing  the  data  (e.g.,  detector  alann  files  of  possible  target  locations)  to  data 
analysts  for  scoring. 

(6)  Test  Support  Personnel:  Prepare  the  lanes  to  remove  all  visual  cues  of  target 
locations,  assist  with  alann  marker  placement  and/or  removal,  and  after  scanning  of  each  lane,  to 
remove  visual  cues  of  operators  and  data  collector’s  foot  prints  (e.g.,  raking,  dragging  the  lane, 
etc.),  and  other  duties  as  needed  to  keep  lanes  ready  for  operators  to  scan  the  test  lanes. 
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3.3  Factors  and  Conditions  for  Perfonnance  Testing. 

The  cumulative  test  results,  together  with  the  results  of  appropriate  engineering  and  data 
collection  tests,  will  allow  an  estimate  to  be  made  of  the  degree  and  readiness  to  which  the 
design  requirements  of  the  detection  system  has  been  met,  and  the  suitability  of  the  system  to 
meeting  operational  need.  The  conceptual  test  assessment  should  cover  the  following  areas  of 
concern: 


a.  Sensitivity.  The  object  is  to  obtain  a  measure  of  the  overall  system’s  ability  to  detect 
targets  by  providing  the  operator  a  reliable  notification  (alarm)  of  when  a  possible  target  is 
within  detection  parameters.  The  sensitivity  assessment  provides  the  ability  of  the  detection 
system  to  detect  metallic  and  non-metallic  targets  as  a  function  of  the  search  head’s  height  about 
the  surface  and/or  the  depth  of  the  target  below  the  surface.  Controlled  calibration  or  data 
collection  lanes  typically  support  this  type  of  assessment. 

b.  Mutual  Interference.  The  objective  is  to  determine  the  minimum  non-interference 
distance  between  two  operating  detection  systems  and/or  with  other  fielded  detection  systems. 
Operate  two  matching  or  other  detector  type  adjacent  to  each  other  and  determine  minimum 
distance  before  operational  interference.  Compatibility  and  susceptibility  assessments  with  other 
tactical  RF  systems  (e.g.,  radios,  radar  systems,  counter  remote  control  improvised  explosive 
devices  (RCIED)  electronic  warfare  (CREW)  systems,  etc.)  are  vital  to  the  ability  of  the  detector 
under  test  to  be  integrated  in  today’s  RF  combat  environment. 

c.  Human  Factors.  The  objective  is  to  detennine  the  detector’s  stability  with  operator’s 
handling  of  the  system,  how  detection  system  notifies  the  operator  (e.g.,  audibly  or  visually),  and 
whether  fatigue  plays  a  factor  during  required  mission  operational  time  profile. 

d.  Performance.  The  objective  is  to  obtain  a  measure  of  the  detector’s  ability  to  perform 
its  intended  function  in  simulated  mission  situations.  The  measure  of  ability  is  detennined  by 
Probability  of  Detection  (Pd),  False  Alann  Rate  (FA),  and  Advance  Rate  (AR). 

e.  Environmental.  The  objective  is  to  determine  ability  of  the  detector  to  operate 
effectively  during  and  after  it  has  been  subjected  to  extreme  environmental  and  logistic 
transportability  conditions  (transportation  vibration,  rough  handling,  climatic,  and  situational 
(e.g.,  rain,  immersion,  salt  fog,  fungus,  altitude,  high  and  low  temperatures,  etc.)). 

3.3.1  Test  Variables. 

a.  Testing  shall  be  designed  to  collect  system  performance  data  over  a  variety  of 
ground/terrain/soil  conditions,  time-of-day  considerations,  and  with  various  standardized  threat 
and  clutter  targets.  To  the  extent  possible,  the  trials  will  be  controlled  and  executed  such  that  the 
systems’  performances  are  directly  comparable  to  baseline  systems  to  be  replaced  or  competing 
systems.  Since  climate  cannot  be  controlled  at  outdoor  test  ranges,  the  tester  must  consider  how 
data  between  test  samples  at  different  sites/locations,  time  of  day,  or  time  of  year  can  be 
compared  effectively.  The  tests  will  be  conducted  with  data  element  considerations  outlined  in 
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Table  2.  It  is  recommended  to  benchmark  developmental  and  competing  systems  concurrently 
over  same  lanes,  same  targets,  and  same  environmental  conditions  with  a  rotation  of  operators. 


TABLE  2.  MATRIX  OF  TRIAL  FACTORS,  CONTROLS,  AND  CONDITIONS 


FACTORS 

CONTROLS 

CONDITIONS 

Threat  Target  Type 
(ITOP  04-2-52 11") 

Standardized  Repeatable  Targets 
(unclassified),  Current  Threats, 
Availability,  Authentic  Components 

Actual;  downloaded,  simulated,  surrogate  target 
types. 

Target  Layout 

Current  Intelligence,  Common  Test 
Practices,  Doctrine  Configuration 

Replicate  burial  depth,  surface  preparation; 
emplacement  techniques;  stacking  and/or  cluster 
techniques,  target  proximity  to  clutter,  fill 
techniques,  orientation,  depth  measurement 
techniques. 

Detection  Lanes 

Prepared  Test  Lanes,  Various  Soil 
Media  Types  (Homogenous  and 
Native);  Moisture  Content 
Management 

Homogenous  soil  media:  loam,  sand,  gravel,  mix 
material;  to  include  grass  vegetation,  loose  and 
compacted  ground,  virgin  ground,  controlled  wash 
beds.  Allow  for  a  calibration  to  be  available  for 
each  type  of  background  condition.  Native  soils 
need  characteristic  soil  profiles  data  established. 

Time-Of-Day 

Diurnal  cycle  spans  all  conditions 
of  solar  radiation  and  thermal  input 

Dividing  diurnal  cycles  into  defined  intervals  (e.g., 
Daytime  (1000-1800  his);  Evening  (1800-2200 
hrs);  Night  (2200-0600  hrs);  and  Morning  (0600- 
1000  hrs)). 

Climate 

Ensure  that  various  test  sites  are 
established  or  identified  that 
provide  full  range  of  climates.  Test 
in  those  conditions  required  in 
requirements  document. 

Use  STANAG  43  702  to  characterize  climatic 
conditions  for  specific  test  sites.  Select  test  sites 
based  upon  range  of  conditions:  arid,  tropical, 
frozen,  moderate,  etc.  use  standardized  target  sets  at 
all  sites  selected  for  specific  trial  phases. 

Human  Participants 

Common  scanning  techniques; 
training  skilled  operators. 

Primary  choice  -  select  test  participants  with  the 
correct  or  associated  military  skills  for  operating 
detection  systems  to  include  both  male  and  female 
personnel  wearing  a  full  complement  of  required 
clothing  and  equipment. 

Secondary  choice:  civilians  that  are  trained  to  a 
common  standard  for  target  detection  to  include 
both  male  and  female  personnel  wearing  a  full 
complement  of  required  clothing  and  equipment. 

Detection  System 

Existing  Baseline 

Compare  to  existing  system  to  be  replaced  or 
competitor(s)  to  provide  baseline  reference  (i.e., 
better  than,  etc).  Ensure  that  sufficient  sample  size 
is  available  to  be  a  production  representative  sample 
or  a  statistically  relevant  population.  Have 
sufficient  spares  or  on-site  field  service 
representatives  for  repairs  to  prevent  test  delays. 
Maintaining  adequate  power  supplies  (e.g., 
batteries)  to  ensure  continuous  scanning  operations. 

*  Superscript  numbers  correspond  to  Appendix  J,  References. 
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TABLE  2.  CONTINUED 


FACTORS 

CONTROLS 

CONDITIONS 

Environmental 
Simulation  Tests 

Climatic  Chamber  Tests 

Test  to  the  specific  environments  identified  as 
mission  essential  requirements  (e.g.,  sand/dust,  hot, 
cold,  rain,  salt  fog,  humidity,  logistic  handling, 
etc.). 

b.  A  matrix  of  conditions  should  be  established  with  a  finite  number  of  threat  and  clutter 
encounters  planned  for  each  condition  (e.g.,  at  each  depth,  in  each  soil,  for  each  target  type,  etc.). 
The  total  number  of  trials  will  drive  the  duration  of  test.  Therefore,  it  is  incumbent  upon  the  test 
planner  to  limit  the  test  conditions  to  a  period  in  which  all  parameters  can  be  practically 
addressed.  The  test  planner  should  ensure  that  for  each  human  participant/system  combination,  a 
corresponding  minimum  number  of  repetitions  of  background,  system  type,  performance  test 
area,  etc.,  occur  to  provide  a  statistically  relevant  set  of  system  performance  trials.  See  Table  3 
for  examples  of  test  variables  to  aid  in  developing  trials  to  enable  assessment  in  a  various  range 
of  factors  and  conditions,  or  a  specific  perfonnance  characteristic  interest. 


TABLE  3.  EXAMPLES  OF  TEST  VARIABLES 


ELEMENT 

CONTROL  FACTORS 

VARIABLE  CONDITIONS 

Detection  Test  Lane 

Tactically  Varied 

Improved,  native  (in-situ),  simulated 
(shipped  in  gravel,  sand,  and  topsoil 
material),  straight,  curved,  dips,  and 

hills 

Target  Type 

Systematically  Varied 

Antitank/antipersonnel  mines, 

Improvised  Explosive  Devices  main 
charges ,  and  trigger  components,  and 
clutter.  Metallic,  low  metallic,  or  no 
metallic  contents,  foreign  and  domestic, 
explosives  or  simulant. 

Target  Placement 

Systematically  Varied 

Depth  buried,  configuration, 
compaction,  multiple  targets,  etc. 

Soil  Type 

Semi-Controlled 

Loam,  sand,  gravel,  desert  silt,  roadway 
beds,  compacted  soil  material 

Climatic  Conditions 

Systematically  selected  but 
uncontrolled 

Standardized  Climatic  Categories13  (e.g., 
Al,  B2,  Cl,  etc.). 

Light  Conditions2 

Systematically  varied,  if 
necessary 

Dawn,  Day,  Twilight 

Operational  Status 

Uncontrolled 

Switching  of  arms;  taking  interim 
breaks  during  detection  run 

“Night  operations  are  not  typical  and  detection  systems  may  not  incorporate  night  visual  capabilities  other  than  the 
use  of  external  support  equipment  (e.g.,  flashlights,  night  vision  optics,  etc.). 

bClimate  Categories  (MIL-STD  810G3):  A1  -  Hot-Dry  Climate,  B2  -  Variable  High  Humidity,  Cl  -  Basic  Cold 
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3.3.2  Other  Considerations. 

a.  Special  Test  Considerations. 

(1)  Operators.  The  operators  must  be  identified  early  in  the  planning  stage  to  whether 
they  are  required  to  be  military  and/or  civilians,  and  where  will  they  come  from  (e.g.,  customer 
provided,  test  range  selection,  etc.).  If  military  personnel  are  required,  ensure  a  Test  Schedule 
and  Review  Committee  (TSARC)  request  is  submitted  within  one  year  from  the  start  of  testing 
or  as  early  as  possible.  A  Safety  Release  (SR)  must  be  obtained  from  the  U.S.  Army  Evaluation 
Center  (AEC)  prior  to  using  military  personnel  as  test  participants.  The  SR  indicates  a  system  is 
safe  for  use  and  maintenance  and  describes  the  specific  hazards  of  the  system,  operational  limits, 
and  required  precautions  (see  Department  of  the  Army  Pamphlet  (DA  PAM)  73- 14).  If  military 
operators  are  not  available,  or  SUT  quantity  exceeds  required  number  of  operators,  civilian 
operators  (range  support  civilian  personnel)  are  often  used  to  supplement  the  required  number  of 
operators.  It  is  preferred  that  skilled  personnel  with  mine  sweeping  training  be  primary 
candidates,  but  not  necessary.  All  personnel  will  be  undergo  a  training  period  prior  to  testing  to 
ensure  there  are  common  scanning  techniques,  and  the  testing  methodology  is  standardized  for 
all  operators.  To  ensure  a  full  spectrum  of  possible  operators,  operators  should  include  both 
male  and  female  participants  wearing  a  full  complement  of  required  clothing  and  equipment. 

(2)  Special  Test  Controls.  There  may  be  special  test  controls  added  to  ensure  that 
operators  use  detection  equipment  under  operationally  realistic  conditions.  Since  it  may  not  be 
practical  or  exceedingly  costly  to  build  numerous  test  lanes  with  actual  threat  targets,  it  may  be 
necessary  to  design  the  test  so  that  operators  do  not  ‘Team  or  game”  the  test  lanes.  Test  controls 
should  include:  training;  pilot  tests  or  trials  in  the  Sensor  Verification  Area  (e.g.,  data  collection 
area);  rotation  of  test  operators;  changes  in  directions,  times  of  day  (see  Table  1),  deliberate  false 
sets  of  targets,  varying  “weathering  in”  of  targets,  and  if  possible,  varied  vegetation  and  soil 
conditions. 

b.  Safety  Documentation. 

(1)  Safety  Assessment  Report  (SAR).  Safety  documentation  on  the  system  under  test 
is  obtained  from  the  materiel  developer’s  safety  team.  The  SAR  is  a  formal,  comprehensive 
safety  report  that  summarizes  the  safety  data  that  has  been  collected  and  assessed  for  the  life 
cycle  of  an  item.  It  expresses  the  considered  judgment  of  the  contractor  or  developing  agency 
regarding  the  hazard  potential  of  the  item  and  any  actions  or  precautions  that  are  recommended 
to  minimize  these  hazards  and  to  reduce  the  exposure  of  personnel  and  equipment  to  them. 
Material  Safety  Data  Sheets  (MSDS)  must  be  provided  for  the  batteries  and  any  other  hazardous 
material. 


(2)  Preliminary  Hazard  Assessment  (PHA).  Broad  hazard-screening  tool  that 
includes  a  review  of  the  handling  and  operation  using  the  system  under  test,  and  identifies  the 
hazards  associated  with  the  operations  on  the  test  range.  The  results  of  the  PHA  are  used  to 
determine  the  need  for  additional,  more  detailed  hazard  analysis,  serve  as  a  precursor 
documenting  that  further  analysis  is  deemed  necessary,  and  serve  as  a  baseline  hazard  analysis 
where  further  analysis  is  not  indicated. 
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(3)  Safety  Confirmation  indicates  if  specific  safety  requirements  are  met,  includes  a 
risk  assessment  for  hazards  not  adequately  controlled,  lists  technical  or  operational  limitations  or 
precautions,  and  highlights  safety  problems  that  require  further  investigation.  The  Safety 
Confirmation  is  required  to  permit  the  system  for  tactical  deployment. 

(4)  Technical/Operators  Manuals  contains  the  manufacturer’s  procedures,  as  well  as, 
the  cautions  and  warnings  when  handling  and  operating  the  SUT. 

(5)  Range  Standard  Operating  Procedures  from  test  range  reviews  of  the  system, 
listing  hazard  mitigating  steps  for  the  operator  of  the  SUT  and  test  personnel. 

c.  Training. 

(1)  Training  is  required  to  orientate  operators  with  the  system(s)  under  test  and  proper 
scanning  techniques.  Training  also  includes  communication  orientation  between  operators  and 
data  acquisition  personnel  to  develop  the  verbal  link  from  operator  to  data  collection  personnel 
for  documenting  operational  activities.  Assure  that  the  proper  training  for  the  operation  of  the 
system  is  provided  to  the  testers,  operators,  and  if  necessary,  the  data  collectors.  It  is  imperative 
that  the  emergency  shutdown  procedures  are  emphasized  in  this  training,  which  could  be  no 
more  than  the  OFF  switch.  An  operator  must  be  well  trained  in  what  to  do  should  a  problem 
occur  and/or  if  a  problem  is  reoccurring.  Operators  and  test  team  training  must  always  be  the 
lead-in  activity. 

(2)  Following  training,  a  pilot  test  should  be  performed  to  ensure  operator  and  test 
team  familiarity  with  the  SUT  and  test  methodology,  which  may  include  performing  a  number  of 
practice  detection  scans  on  calibration/data  collection  lanes.  Proper  scanning  techniques  and 
coverage  procedures  should  be  stressed  during  this  training  and  pilot  period.  Pilot  tests  will  also 
serve  to  familiarize  detector  operators  and  test  observers  (data  collectors)  with  the  data  collection 
and  reporting  procedures  to  be  employed  during  system  performance  trials. 

NOTE:  Performance  of  Detection  tests  are  those  trials  where  data  will  be  used  towards 
assessing  the  effectiveness  of  system  detection  performance,  or  for  other  evaluation  purposes 
relative  to  the  suitability  of  the  detection  system.  Tests  should  not  begin  until  each  team  member 
has  demonstrated  competence.  Test  team  rotation  is  intended  to  ensure  that  lower  or  higher 
performing  individuals  do  not  bias  performance  results.  Deliberate  false  sets  of  targets  (clutter) 
should  be  introduced  to  include  empty  holes,  surface  disturbances,  or  rural  and  battlefield  debris. 
Weathering  in  of  targets  enables  proper  testing  of  the  detection  equipment,  to  include  removing 
visual  cues  that  provide  the  operator(s)  a  secondary  opportunity  to  locate  the  buried  targets. 
Counter  Measure  (CM)  equipment  may  have  to  perform  after  the  targets  have  been  weathered  in 
place  for  1  day  to  1  month.  Demining  equipment  may  have  to  perform  after  the  targets  have 
been  weathered  in  place  for  1  month  or  much  greater.  Pre-test  dragging  of  test  areas/lanes  may 
eliminate  soil  settling  issues  around  buried  target  locations. 

d.  Operators  interaction  with  data  collectors.  Communication  between  operator  and  data 
collectors  during  detection  trials  must  be  refined  prior  to  the  start  of  formal  testing.  Time  data 
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(e.g.,  detector  operational  duration,  battery  usage  time,  mission  time,  etc.)  must  be  collected 
during  testing,  and  typically  a  good  time  to  practice  communication  between  operator  and  data 
collector  is  during  training/orientation  periods.  Information  that  must  be  collected  by  data 
collectors  during  testing,  but  not  limited  to,  include:  when  batteries  are  changed,  when  detectors 
are  switched  OFF/ON,  when  detection  alanns  are  activated,  confidence  level  of  operator  related 
to  actual  target  detection  (false  alann,  or  not),  and  any  other  anomalies  while  perfonning  the 
detection  mission  on  the  test  lanes.  The  operator  must  constantly  verbalize  information  to  the 
data  collector  who  is  documenting  what  is  being  declared  in  test  log  books. 

e.  System  Tracking/Monitoring  Database.  Prior  to  the  start  of  testing,  data  tracking 
recording  methods  on  the  system  under  test  may  be  established.  The  test  team  must  monitor  the 
system  as  a  whole,  as  well  as  individual  components  and  support  systems  (e.g.,  batteries,  straps, 
etc.).  Proper  titles/names  of  components  must  be  obtained  and  verified  with  materiel 
developer/contractor  to  be  compatible  with  technical/operational  manuals.  Data  elements  must 
be  outlined  with  specific  units  identified  in  test  plans.  Performance  parameters  and  scoring 
criteria  should  be  identified  to  assist  in  the  documentation  to  reflect  preliminary  success  or 
limitations  experienced  during  testing.  Also,  failures  or  discrepancies  of  test  items  are 
documented  in  the  tracking/monitoring  database  to  assist  in  reliability  assessment. 

3.4  Frequency  Allocation. 

a.  The  tester  is  responsible  for  requesting  radio  frequency  (RF)  assignments  from  the 
local  spectrum  frequency  manager  in  the  proposed  area  of  testing.  Dates  required  for  the  entire 
test  period,  plus  a  contingency  period  if  testing  delays  are  experienced,  should  be  included  in  the 
request.  The  local  Spectrum  Management  Office  (SMO)  receiving  the  frequency  assignment 
application  (DD1494  form  and  or  J5)  will  process  it  through  established  channels  to  request 
appropriate  national  and  local  approvals.  The  disposition  of  the  frequency  assignment  request 
will  be  channeled  back  to  the  tester.  The  request  may  be  approved,  disapproved  in  total,  or 
approved  in  part  with  operating  limitations  (notching  out  specific  frequencies).  Typically, 
frequency  approval  takes  about  50  days,  but  resolving  conflicts  can  take  up  to  six  months  lead 
time,  so  this  infonnation  must  be  requested  and  submitted  as  early  as  possible.  Appendix  A  lists 
the  infonnation  needed  to  submit  a  request  on  the  DD1494  form. 

b.  This  RF  infonnation  must  be  provided  by  the  test  proponent  no  less  than  eight  weeks 
prior  to  the  start  of  testing  in  order  to  ensure  approvals  are  obtained  to  meet  the  desired  test 
schedule.  This  information  must  be  provided  for  each  transmitter  and  receiver  of  a  particular 
system  being  used  for  test  and  in  support  of  testing  (fielded  unit  used  for  comparison). 

3.5  Other  Supporting  Documents. 

a.  Performance/requirements  specifications  contain  complete  design  and  operational 
specifications  for  the  system  requirements.  It  includes  the  series  of  targets  that  the  SUT  is 
required  to  detect,  and  what  the  mission  role  of  the  intended  operating  units,  as  well  as  other 
users’  need  for  the  system.  In  addition  to  operational/functional  requirements,  the 
performance/requirement  specifications  should  also  contain  nonfunctional  (or  supplementary) 
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requirements  which  impose  constraints  on  the  design  (such  as  quality  standards  and  design 
constraints). 

b.  Capabilities  Requirements/Operational  Requirements  Document.  Describes  the 
overall  mission,  the  type  of  system  proposed  and  the  anticipated  operational  concepts  in 
sufficient  detail  for  program  and  logistics  support  planning,  and  includes  a  brief  summary  of  the 
mission  need.  This  type  of  document  would  include  criteria  that  the  system  must  meet. 

c.  Security  Classification  Guide  (Required  Document).  Classification  guidance  issued 
by  an  original  classification  authority  that  identifies  the  elements  of  information  regarding  a 
specific  subject  that  must  be  classified  and  establishes  the  level  and  duration  of  classification  for 
each  such  element.  See  Appendix  H  for  example  definitions  of  classified  versus  For  Official  Use 
Only  (FOUOyUnclassified  data  elements  that  must  be  agreed  upon  and  concurred  by  all  parties 
(e.g.,  item/program  manager,  materiel  developer,  testers,  security  etc.)  prior  to  the  start  of 
testing. 

d.  Programmatic  documents  that  may  have  been  developed  for  the  system,  or  those  pre¬ 
existing  documents  that  apply,  should  be  used  to  identify  test  criteria  and  detennine  test 
limitations  and  safety  hazards  that  may  exist.  These  documents  may  define  the  mission 
scenarios,  climatic  conditions,  operational,  and  electromagnetic  environments  in  which  the  item 
must  operate.  In  some  cases,  it  may  define  the  test  center  which  relates  to  the  type  of 
environmental  background  the  test  must  be  executed  in.  Documents  that  should  be  used  to 
establish  test  criteria  include: 

(1)  Test  and  Evaluation  Master  Plan  (TEMP). 

(2)  Test  Directives. 

(3)  Military  Regulations,  Standards  and  other  controlling  directives. 

(4)  Federal/State  Statutes  and  Environmental  Laws. 

(5)  Military  Field  Manuals. 

(6)  Material  Handling  Data  Sheets. 

4.  TEST  PROCEDURES. 

4.1  Receiving/Initial  Inspections. 

Inspections  will  be  used  as  test  controls/baseline  conditions  to  assure  readiness  of  SUT  and  to 
reduce  decision  risk  where  sample  sizes  are  limited. 

a.  The  SUT  will  be  inspected  prior  to  the  commencement  of  testing.  The  initial 
inspections  will  establish  the  status  of  the  SUT  and  supporting  test  equipment  prior  to  the  start  of 
the  tests.  The  results  will  be  recorded  and  serve  as  the  test  control.  The  inspection  should 
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establish  that  the  SUT  is  complete  with  ah  required  operational  components  and  software,  and  is 
in  a  safe  working  order.  It  is  to  be  checked  for  damage,  and  if  it  conforms  to  specifications. 

b.  Initial  inspections  will  document  the  packaging,  nomenclature  (manufacture  serial 
numbers  correlating  to  test  center  induced  identifying  numbers),  type  (if  special  variants  are 
built),  and  quantity  inventory  of  each  type  of  test  item.  Inspections  may  consist  of  a  combination 
of  physical  and  non-destructive  examination  to  detennine  the  safety  of  the  item  and  the  physical 
condition  prior  to  and  following  tests.  Inspections  may  be  augmented  by  other  forms  of 
electronic  checks  with  special  test  instrumentation  if  test  hardware  has  been  designed  to  facilitate 
this  type  of  check. 

The  test  team  must  perform  a  characterization  inspection  of  the  detection  SUT(s)  and 
take  photographs  showing  overview  of  test  item(s),  images  at  each  orientation  (e.g.,  360  degree 
around),  specific  and  unique  components,  accessories,  cables,  connectors,  markings,  etc.  This 
aids  in  reviews  and  analysis  of  items  when  investigating  specific  areas  of  interest.  Photograph 
ah  damage  or  visual  discrepancies  to  document  condition  at  the  time  of  inspection.  (NOTE: 
testers  can  never  have  too  many  pictures  or  views  of  a  test  item.)  It  is  recommended  to  take 
photographs  of  test  item  in  isolated  view  (no  other  objects  in  view  frame),  ah  orientations,  and 
with  operator(s)  in  control  of  test  item  (in  ah  human/machine  interface  configurations).  This 
provides  a  resource  of  images  to  pick  from  or  assess  during  follow-on  test  operations  or 
reporting  activity.  It  also  provides  a  visual  description  of  operator/hardware  integration,  as  well 
as  assessing  for  human  factors  engineering  and  safety. 

c.  Technical/Operator  Manuals  (TM/OM).  Obtain  the  latest  version  of  the  manual. 
Document  what  version  is  provided  (edition  and  date).  TM/OM  is  reviewed  to  ensure  it 
addresses  the  specific  configuration  of  hardware  and  software  under  test,  and  the  accuracy  of 
information  provided.  The  TM/OM  must  be  reviewed  for  safety  warning,  hazards,  precautions, 
and  operational  procedures.  Document  any  discrepancies  with  test  agency  and  materiel 
developer.  Ah  safety  or  operational  discrepancies  should  be  reviewed  with  materiel  developer 
prior  to  test  hardware  activation  or  start  of  test  operations,  depending  on  the  severity  of 
discrepancy. 

d.  Perform  a  safety  inspection  by  a  qualified  safety  engineer  or  test  officer  to  identify 
potential  safety  and  health  hazards  on  the  SUT  prior  to  the  start  of  functional  operations.  Induce 
safety  mitigations,  if  necessary,  to  prevent  hazards  to  operator. 

e.  Preventative  Maintenance  or  Assembly  Requirements.  If  applicable,  complete  a 
preventive  maintenance  checks  and  services  (PMCS)  action  in  accordance  with  the  TM/OM. 
Often,  a  Field  Service  Representative  (FSR)  (associated  with  materiel  developer  or  developing 
contractor)  must  perform  pre-test  tasks  to  assemble  the  item(s)  prior  to  officially  handing  over 
the  test  hardware  for  test.  After  the  completion  of  the  assembled  hardware,  perform  an  initial  or 
another  characterization  inspection. 

f.  Operational  Check.  Test  items  must  be  powered  on  and  operated  to  verily  operational 
status  at  the  time  of  receipt/inspection.  Test  personnel  must  utilize  the  required  power  source 
(e.g.,  battery,  voltage,  etc.)  to  activate  the  system.  Electronic  Built-in-Test  (BIT)  are  performed, 
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sensors  are  activated  with  simulated  targets  (source  of  detector  stimuli),  and  displays,  lights,  and 
light-emitting  diodes  (LEDs)  are  checked  to  verify  operational  status.  This  operational  check  is 
the  baseline  to  demonstrate  the  system  under  test  is  operationally  ready  for  test. 

g.  Inventory  the  system  support  package  (SSP),  if  provided  (e.g.,  straps,  nuts  and  bolts, 
covers,  etc.).  Record  the  findings  of  that  inventory. 

h.  Compliance  to  the  test  criteria  should  be  limited  to  visual  inspections  or  functional 
checks  using  procedures  given  in  the  supplied  TM/OM,  or  instructions  provided  by  the  materiel 
developer’s  or  developing  contractor.  Operational  functionality  and  physical  characteristics  may 
be  necessary  to  address  whether  SUT  meets  design  and/or  contractual  specifications/criteria. 

4.2  Physical  Characteristics. 

Measure  and  record  the  physical  characteristic  of  the  SUT  (e.g.,  dimensions,  weights,  center  of 
gravity,  etc.).  Check  the  data  during  the  inspection  to  ensure  that  they  meet  the  design  and/or 
contractual  specifications/criteria.  Perform  physical  measurements  of  test  items  in  shipping 
configuration,  bare,  folded/extended,  with  and  with  power  supplies,  etc. 

a.  Size  (of  complete  unit  or  each  subunit,  as  appropriate). 

b.  Weight  (of  complete  unit  or  each  sub-unit,  as  appropriate):  total  weight,  balance 
distribution  (human  factors  issue);  weight  of  power  supply  (e.g.,  battery). 

c.  Volume  (space  utilization). 

d.  Detection  head  size  and  static  area  of  coverage. 

4.3  Performance. 

4.3.1  General. 

a.  All  procedures  for  the  detection  assessment  apply  to  system  perfonnance  trials. 

System  performance  trials  are  designed  to  demonstrate  the  detection  equipment  perfonnance,  as 
well  as,  the  participants’  respective  capabilities  to  detect  buried  targets  in  operationally  realistic 
conditions.  Performance  and  reliability  parameters  will  be  developed  from  the  system 
performance  trials  for  the  purposes  of  detennining  overall  assessment.  Other  sensor  verification 
testing  activities  may  support  source  selection,  perfonnance  benchmarking  or  investigation, 
diagnostics,  data  collections,  training,  or  pilot  and  assessment  trials.  While  these  trials  may  yield 
useful  parameters  for  the  objected  purposes,  data  should  not  be  aggregated  with  the  system 
performance  trials.  Ensure  that  trials  include  the  baseline  fielded  system  for  comparison 
purposes  in  addition  to  the  detection  equipment  candidate(s).  Ensure  that  the  baseline  system  is 
in  new  or  like-new  operational  condition. 

b.  Performance  benchmarking  implies  establishing  a  reference  performance  level  for 
ideal,  controlled  conditions  so  that  a  range  of  expected  perfonnance  levels  and  expected 
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degradations  can  be  established  for  sets  of  ideal,  controlled  conditions.  Additionally,  the 
benchmarking  can  be  accomplished  using  the  system  to  be  replaced  as  a  special  reference 
standard  for  a  set  of  ideal,  controlled  conditions.  While  not  mandatory,  benchmarking  can 
provide  useful  information  for  subsequent  diagnostics,  as  well  as  enabling  side-by-side 
comparisons  with  existing  or  competing  systems  in  controlled  conditions,  especially  when  a 
standard  set  of  targets  are  used. 

c.  Detection  system  testing  should  progress  logically  and  systematically  from  the 
experimental  through  the  system  performance  trials.  Experimental  and  system  perfonnance 
trials  phases  need  to  be  planned  together  to  be  synergistic  and  support  each  other.  Progression 
through  the  test  program  should  provide  increasing  confidence  that  the  capabilities  and 
limitations  of  the  equipment  under  test  are  defined  and  understood. 

d.  The  levels  of  performance  required  of  the  SUT  should  be  specified  by  the  user  in  the 
user  requirements  document  or  the  equipment  requirement  specification.  If  testing  for  urgent 
fielding,  the  detection  targets  and  lane  characteristics  should  be  similar  to  theater  environmental 
conditions. 

4.3.2  Sensor  Experimentation  Testing  (Source  Selection,  Benchmarking,  or  Investigative 
Experimental  Trials). 

a.  Prior  to  the  start  of  all  testing  of  each  SUT  or  baseline  system,  ensure  that  no  targets 
have  been  inadvertently  disturbed.  Record  the  characteristics  and  location  of  each  target’s 
positions  and  burial  depths/orientations.  Ideally,  mark  the  target’s  position  with  non-interfering 
markers  (e.g.,  nonmetallic  devices  -  golf  tees)  that  identify  target  type,  location,  and  depth. 

b.  Mount  the  SUT  detection  system  on  a  maneuverable  platform  above  the  area 
containing  a  target  or  maneuver  using  a  human  participant.  Either  through  cabling,  telemetry,  or 
other  conventional  manual  means  record  the  output  history  as  the  candidate  detection  system 
advances  over  the  target  area.  Ensure  that  advance  rate  and  sensor  height  replicates  the  expected 
heights  and  advance  rates.  If  automated  means  are  used,  record  the  output  history  as  a  function 
of  soil  type,  target  type/location,  path,  advance  rate,  height,  and  candidate  or  fielded  detection 
system.  Other  variables  of  importance  may  be  added. 

c.  Depending  upon  sensor  outputs  and  signal  processing/display  methods,  the  start  and 
stop  times  for  each  “trial”  or  “mission”  should  be  recorded  to  establish  a  time  history  for  the 
trial.  This  will  also  detennine  the  advance  rate  if  required. 

d.  During  and/or  following  each  trial,  the  location  data  (e.g.,  detection  alarm)  for  each 
detection  mark  should  be  recorded  and  compared  to  each  buried  target  location  to  determine  if 
the  declaration  “alarm”  spot  would  have  been  a  detection  event  or  false  alarm.  At  this  sensor 
experimental/development  facility,  for  most  candidate  detection  systems,  false  target  sets  or 
disguised  target  sets  (clutter)  are  optional,  and  may  not  be  of  interest  or  desire,  where  focusing 
strictly  on  valid  targets  is  the  primary  concern. 
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e.  A  description  of  the  Sensor  Experimentation  Testing  Facility  should  be  included  with 
the  following  documented  conditions:  soil  types,  soil  moisture/density,  test  area  size,  buried  or 
uncovered,  natural  or  artificial  light,  ambient  or  artificial  climate  control,  and  other  descriptors. 
Other  soil  properties  may  be  of  value  for  specific  sensor  types.  If  artificial  light  or  artificial 
climate  control  is  used,  the  range  of  values  occurring  during  test  should  be  recorded. 

4.3.3  Sensor  Verification  Testing  (Data  Collection  Trials). 

Sensor  verification  testing  should  be  done  initially  by  special  test  technicians  or  materiel 
developer’s  subject  matter  experts/contractor  operators  upon  receipt  of  candidate  or  fielded 
detection  system(s).  This  testing  will  be  performed  to  determine  if  the  test  items  are  ready  for 
test  or  have  been  damaged  as  a  result  of  shipment.  After  initial  inspection  and  sensor  validation 
the  Sensor  Verification  Test  Area  will  be  used  for  training,  demonstration  and  pilot  tests 
(learning  trials),  and  diagnostics  (performance  verification  and  troubleshooting  software  faults). 

a.  The  sensor  verification  area  will  be  set  up  with  geodetically  surveyed  detection  targets 
of  all  types  identical  to  those  being  used  during  formal  system  performance  testing.  Record  the 
geodetic  surveyed  plot  of  target  positions  and  burial  depths/orientations.  Photographs  should  be 
taken  of  target  placement  prior  to  backfilling.  The  ground  truth  database  documenting  the 
coordinate  location  and  target  characteristics  is  established,  maintained,  and  updated 
continuously.  Prior  to  the  start  of  all  trials,  inspections,  training,  or  pilot  test  of  each  candidate  or 
fielded  system,  the  test  team  must  ensure  that  no  target  has  been  inadvertently  disturbed. 

b.  Document  inspections,  training,  or  pilot  tests  to  assure  test  readiness  and  provide  a 
historic  tracking  of  developmental  progress.  This  data  (derived  from  standardized  data  sheets) 
would  be  entered  into  the  Tracking/Monitoring  database  (paragraph  3.2.e).  This  is  a  good 
opportunity  to  collect  upfront  assessments  concerning  safety,  human-machine  interface,  as  well 
as,  detection  perfonnance.  It  also  documents  the  hardware  readiness.  Diagnostics  should  also  be 
documented  to  provide  an  audit  of  readiness  issues,  factors,  or  recommendations  that  should  be 
corrected  prior  to  formal  scoring  tests  or  even  fielding.  Examples  may  include  corrected  field 
manuals,  training  packages,  scanning  techniques  or  procedures,  or  amending  test  procedures. 

c.  Pilot  tests  in  the  Sensor  Verification/Data  Collection  test  areas  should  parallel  the  test 
process  and  methods  that  will  be  used  during  fonnal  performance  tests.  This  will  establish  test 
execution  proficiency  to  the  entire  team  of  test  participants  (test  officers,  operators,  and 
representatives)  and  test  observers  (data  collectors)  prior  to  system  performance  trials.  It  also 
provides  a  rehearsal  session  between  the  test  participants/operators  and  data  collectors  to  adjust 
the  collection  of  required  data  elements  during  the  scanning  and  detection  process,  so  when 
formal  testing  is  executed,  the  data  collectors  are  transparent  to  the  test  participants/operators 
without  hindering  their  mission  (e.g.,  operators  are  verbalizing  information  about  the  detector 
and  detection  performance  to  the  data  collector). 

4.3.4  System  Performance  Trials  Test  Lanes/ Area  (Blind  Perfonnance  and  Scoring). 

4.3.4. 1  Pretest. 
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a.  Prior  to  start  of  test,  ensure  that  all  test  readiness  issues  have  been  resolved 
satisfactorily  (e.g.,  detection  systems  are  latest  configuration  and  operational,  test  areas  prepared 
and  conditioned  for  detection  scanning,  operators  are  orientated  with  detection  systems,  etc.). 
Each  detection  test  lane/area  will  be  set  up  with  surveyed  targets  of  all  types  to  meet  threats  that 
are  common  to  existing  military  conflicts  or  terroristic  activities  (e.g.,  IED  configurations  for  a 
given  region  of  the  world).  However,  these  target  representatives  must  be  agreed  upon  by  the 
acquisition  assessment  chairman  and  materiel  developer  and  meet  threat  target  objects  (e.g., 
metallic,  low  metallic,  and  non-metallic  target  types  and  configurations).  All  targets  must  be 
surveyed  during  emplacement  using  Differential  Global  Positioning  System  (D-GPS)  survey 
equipment  with  accuracy  to  ±  2  centimeters  (cm)  to  obtain  coordinates  in  Universal  Transverse 
Mercator  (UTM)-World  Geodetic  System  (WGS)84  grid  prior  to  burying  (covering/disguising). 
Record  the  plot  of  mine  positions  and  burial  depths/orientations  along  with  target  identification 
(ID)  and  corresponding  characteristics.  A  photograph  of  each  target  prior  to  burying  aids  the 
analysis  team  in  target  identification  when  any  questionable  issues  surface  about  a  specific  target 
position.  Ensure  the  test  lanes/areas  are  spread  out  and  separated  enough  to  allow  for  multiple 
systems  to  interrogate  lanes  simultaneously  without  interfering  with  each  other  (e.g.,  electromagnetic 
interference). 

b.  The  test  lanes/area  will  be  operationally  realistic.  When  clutter  or  ground  disturbances 
are  added  (false  target  sets,  vegetation,  etc.),  these  will  be  surveyed  to  obtain  coordinates, 
photographed,  and  narrative  description  recorded.  Each  test  lane/area  will  be  assigned  a  unique 
test  ID  number  (e.g.,  Lane  ID).  Each  planned  trial  or  mission  will  have  a  unique  mission  ID. 
Each  test  player  will  have  a  unique  test  player  ID  to  be  retained  throughout  testing.  If  teams  of 
test  players  are  used  for  a  search  method,  the  team  will  have  a  unique  ID. 

c.  Test  Schedule  and  Daily  Events.  The  perfonnance  test  phase  must  include  several 
phases/events  that  are  part  of  the  overall  test  effort.  These  phases/events  must  be  addressed  and 
scheduled  time  allocated  for  an  effective  test  assessment  (e.g.,  allocating  time  for  operator  and 
test  team  system  training,  test  methodology  orientation,  execution  plan  to  accomplish  the  test 
matrix,  and  intennittent  periods  for  Human  Factors  assessment). 

(1)  Training/Orientation  Time.  Training/orientation  periods  should  be  allocated  as 
close  to  the  start  of  the  detection  mission  trials.  Detailed  training  and  periodic 
orientation/refresher  training  on  the  systems  under  test  must  be  programmed  for  operators  and 
test  personnel,  especially  if  rotational  between  different  types  of  SUTs.  Training  should  be 
geared  to  the  lowest  level  of  operator  skills.  Data  collectors  and  test  personnel  may  not  be  as 
advanced,  and  may  only  need  a  thorough  overview  of  the  system(s)  under  test;  whereas  operators 
need  hands-on/sweep  time  to  fully,  effectively,  and  proficiently  operate  the  systems  under  test. 
There  may  be  several  categories  of  operators:  subject  matter  experts  (SME),  military 
experiences,  and  novice  (beginners).  Depending  on  the  level  of  assessment,  a  mix  of  operator 
types  provides  an  overall  assessment  of  the  system  human-machine  interface  (e.g.,  ease  of 
operation,  interruption  of  alarm  indications,  scanning  techniques,  etc.). 

(a)  Training  should  be  provided  by  a  trainer  who  was  formally  trained  by  or 
represents  the  manufacturer  on  that  specific  item  under  test. 
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(b)  Training  is  conducted  on  target  types  and  detector’s  response  to  metallic  and  non- 
metallic  signatures  along  calibration  devices  (positive  sensor  responses).  This  provides  the 
control  and  audible  functions  and  target  determination  the  detector  provides  when  activated  by  a 
valid  target.  Operators  should  be  provided  instruction  on  a  calibration  lane,  area  where  targets 
are  known:  location,  type,  and  depth.  Operators  will  be  instructed  over  training  calibration  and 
data  collection  lanes  containing  similar  targets  as  blind  test  lanes.  On  training/calibration  lanes, 
operators  will  be  instructed  on  how  to  perfonn  a  proper  scan  to  maintain  a  consistent  speed  and 
sensor  height  over  area  to  be  scanned,  as  well  as,  to  ensure  the  entire  lane/area  is  covered  by  the 
detection  scanning  process,  as  the  operator  progresses  through  the  lane/area. 

(c)  The  refresher  orientation  is  for  operators  who  have  been  trained  previously  on  the 
system  under  test.  This  gives  them  a  short  review  of  the  operator’s  manual,  overview  of  detector 
system  controls,  and  hands  on  operations. 

NOTE:  Using  expert  operators  helps  to  insure  that  the  detection  systems  are  used  in  an  optimal 
fashion,  assuring  that  the  essential  sensor  data  is  of  high  quality,  and  provides  a  measure  of  the 
higher  confidence  on  system  performance.  Using  independent,  novice,  but  trained  operators, 
provides  a  measure  of  system  performance  that  may  be  similar  to  that  of  typical,  trained  military 
operators. 


(2)  Test  Matrix.  A  test  matrix  (detailed  daily  schedule)  must  be  developed  for  system 
performance  missions  to  ensure  that  all  combined  possibilities  are  addressed  (e.g.,  using  all 
operators,  detection  systems,  and  scanning  over  all  test  lanes  that  contain  required  target 
encounters).  This  planned  matrix  should  be  outlined  in  the  test  plan.  Detailed  mission  run 
specific  elements  (e.g.,  lane  direction,  which  operator  with  what  system,  assigned  data  collector, 
etc.)  are  provided  when  the  tester  outlines  daily  mission  run  activities  to  correspond  to  the 
planned  test  matrix.  The  test  duration  (number  of  test  days  required)  should  be  developed  based 
on  what  the  matrix  timetable  that  can  be  completed  in  a  day’s  work  period  times  the  number  of 
lanes  that  can  be  scanned  in  one  day.  Contingency  days  must  be  planned  for  weather  delays, 
hardware  discrepancies,  or  any  other  uncontrollable  or  unplanned  conditions  or  events. 

4. 3. 4.2  Mission  Detection  Performance  Test  Phase. 

The  generic  methodology  of  how  the  detection  performance  test  is  executed  is  outlined  in  the 
following  paragraphs.  The  execution  phase  is  divided  into  the  following  phases: 

•  Pre-mission  briefs 

•  Detection/Performance  missions 

•  Post-mission  detection  alarm  compilation  and  Quality  Assurance/Quality  Control 
(QA/QC)  checks 

•  Detection  data  scoring  (performance  analysis) 

•  Human  Factors  Operator  Surveys 

a.  Pre-Mission  Briefs.  The  test  officer  provides  a  test  brief  outlining  safety  issues  and 
proposed  mission  matrix  runs  to  all  personnel.  During  the  brief,  the  test  officer  identifies  a 
mission  lane  assignment  (Figure  1)  for  each  operator  and  paired  with  a  data  collector.  The  lane 
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assignment  is  a  unique  identifier  for  that  run.  Data  collectors,  geodetics  surveyors,  and  data 
analyst  personnel  use  the  unique  lane  assignment  to  characterize  the  detection  results  for 
reporting  purposes.  Figure  1  is  an  example  of  one  method  to  identify  a  mission  run.  Other  codes 
could  be  added  as  necessary. 


System 

Run  Number 

1M 

91A  001 E 

Operator 

Lane  Identifier  Scan  Direction 

Figure  1.  Example  of  a  format  for  assigning  test  mission  identification. 

NOTE:  Following  is  a  description  of  each  code  in  the  Test  Mission  ID  elements  (figure  X): 

•  Operator  -  Distinguished  by  the  lead  number.  Each  operator  has  an 
identifying  operator  number. 

•  System  -  If  there  are  multiple  systems  under  test,  each  is  distinguished  by 
using  a  letter  code  (e.g.,  L,  M,  N,  O,  etc.). 

•  Lane  Identifier  -  Blind  test  lanes  are  distinguished  by  number  and  letter 
code:  (e.g.,  91A,  72B,  65,  etc.). 

•  Run  Number  -  Three  digits  are  used  for  run  (mission)  number. 

•  Scan  Direction  -  Referred  to  the  scan  direction  of  travel:  north  (N),  south 
(S),  east  (E),  and  west  (W). 

b.  Detection/Performance  Missions  Events. 

(1)  System  Calibration/Ground  Balancing.  At  the  start  of  each  test  day,  perform  the 
startup  and  operational  procedures  per  OM  or  manufacturer  provided  procedures.  Power  up  the 
detection  system.  Warm  up  and  calibrate  systems  in  accordance  with  (IAW)  startup  procedures 
and  calibration  detection  pieces.  Document  any  failures  found  by  the  BIT  or  other  means,  or  any 
failure  of  the  BIT  itself,  including  false  BIT  indications.  For  pre-and  post-mission  checkouts, 
perform  a  BIT  before  and  after  each  mission.  Record  the  results  on  a  daily  data  log  sheet. 

(2)  System  performance  detection  missions.  The  basic  detection  mission  consists  of 
target  encounters  over  a  test  lane/area  with  the  detection  system  under  a  specified  set  of  test 
conditions.  A  system  performance  mission  (scanning  one  lane)  will  provide  a  finite  number  of 
planned  target  encounters  for  each  path/direction  and  target  type  through  the  test  lane/area. 
Testing  should  be  limited  to  a  specific  number  of  days  and  encounters.  Planned  target 
encounters  must  be  executed  within  this  time  period  because  additional  days  may  not  be 
available  (e.g.,  personnel  and  equipment  availability).  Total  test  days  must  ensure  target  types 
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meet  the  minimum  number  of  engagements  (encounters)  to  support  the  specifications  or 
confidence/probability  goals. 

(3)  Scoring  lane  runs.  Sufficient  time  must  be  allocated  daily  to  survey  the 
completed  mission/performance  runs.  Alarm  target  markers  should  not  be  left  un-surveyed 
overnight.  Data  needs  to  be  collected  and  recorded  by  day’s  end.  Preliminary  scoring  of 
performance  should  be  achieved  on  a  semi-real  time  bases.  However,  adequate  number  of  days 
must  be  allocated  to  process  and  final  performance  data. 

(4)  Data  reports  may  be  required  for  both  classified  and  unclassified  levels  of 
information.  Time  must  be  allocated  in  the  overall  schedule  to  sequential  draft  and  publish  two 
separate  documents  if  required. 

c.  Following  each  day’s  daily  planning  session  outlining  test  requirements,  matrices,  and 
goals,  the  test  team  will  move  out  to  the  test  lanes  for  performance  testing.  The  test  officer 
assigns  each  operator/system  a  blind  test  lane  to  perfonn  a  detection  mission  run.  Sequence  of 
activities  is  as  follows: 

(1)  At  the  start  of  each  test  day,  the  operator  and  data  collector  will  acquire  a  new  set 
of  or  freshly  charged  batteries  to  install  into  the  detector.  The  operators  will  turn  on  detection 
systems  at  the  beginning  of  each  test  lane,  and  perfonn  startup  ground  balancing  and  calibration 
procedures.  Depending  on  assessment  for  reliability,  the  systems  will  be  turned  off  or  left  on 
between  detection  runs. 

(2)  Record  battery  usage  times,  lane  sweep  start  and  stop  times,  and  other  mission 
information  for  each  detection  area  to  be  swept.  An  example  of  a  Mission  Field  Data  Collection 
form  is  provided  in  Appendix  D.  The  form  would  be  completed  by  data  collectors  (or 
equivalent)  that  are  assigned  to  follow  the  detector  operator  during  performance  detection 
activities. 

NOTE:  Execution  of  Performance  testing  is  on  clean  lanes  without  target  markers  or  visual  cues 
(e.g.,  wires,  burial  spots,  target  exposures,  accumulation  of  footprints,  accumulation  of  gravel  or 
ground  material  from  raking  ground  area,  etc.).  Test  team  must  ensure  these  visual  cues  are 
removed  before  any  test  runs.  Typically,  the  lanes  are  cleared  and  dragged  at  the  end  of  a 
detection  scan. 

NOTE:  Most  human  operators  will  deliberately  or  unwittingly  venture  to  locate  and  identify 
targets  for  follow  on  detection  runs.  To  prevent  the  test  participants  from  being  cued  by  prior 
knowledge  for  these  sequential  detection  runs,  special  test  controls  should  be  adhered  to  and 
controlled  by  the  test  director  (e.g.,  dragging  lanes,  removing  debris  or  rocks  landmarks  on  side 
of  lanes,  having  operators  run  in  opposite  directions,  rotation  of  operators,  etc.). 

(3)  Per  the  planned  test  matrix,  perform  detection  mission  scenarios  scanning  the 
entire  designated  test  lane  by  traversing  each  blind  test  lane  sweeping  forward  and  back  until  the 
desired  length  and  width  are  covered  by  the  scanning  process.  Each  operator  will  sweep  the 
designated  area/lane  and  indicate  where  an  alarm  marker  is  to  be  placed  (e.g.,  in  direct  response 
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to  detection  system  alarm  indicator).  The  operator  will  mark  all  suspected  locations  indicated  by 
the  detector’s  audible  and/or  visible  alarm  designator.  An  alarm  marker  (e.g.,  plastic  poker  chip 
or  equivalent)  should  be  placed  on  the  spot  where  the  operator  determines  the  detection  system  is 
providing  the  most  confident  indication  of  a  possible  target’s  location.  Some  assessment  teams 
may  use  different  color  markers  to  identify  high  confidence  detection  location  alarms  and 
another  marker  for  low  confidence  or  suspected  clutter  positions;  however,  this  assessment  is 
highly  operator  subjective  in  nature. 

NOTE:  Detection  mission  runs  are  performed  to  accumulate  the  required  number  of  target 
encounters  distributed  over  the  blind  test  areas/lanes  established  for  this  test.  In  the  matrix 
mission  run  development,  the  goal  is  to  obtain  equal  number  of  encounters  for  all  target  types. 

(4)  This  detailed  process  of  the  detector  system  moving  down  the  lane,  the  operator 
or  a  data  collector  puts  down  detection  alann  markers  (e.g.,  numbered  poker  chips)  where  the 
system  detects  and  the  operator  declares  a  potential  threat  target.  This  event  will  leave  a  trail  of 
poker  chips  down  the  lane.  The  test  support  personnel  will  follow  taking  inventory  of  the  alarm 
markers.  The  numbered  markers  (poker  chip)  are  recorded  on  the  data  sheet  (see  Appendix  D 
for  a  sample  data  sheet). 

d.  Post  Mission  Quality  Checks. 

(1)  At  the  completion  of  scanning  the  lane,  the  data  collector  confirms  the  data  sheet 
count  of  poker  chips  to  the  last  numbered  poker  chip  marker  on  the  lane.  The  number  of 
recorded  detection  events  and  number  of  poker  chip  alarm  markers  should  match.  This  quality 
check  of  alarm  marker  count  is  verified  again  by  comparing  the  number  of  surveyed  alann  poker 
chip  markers  when  the  surveyor  locates  each  chip  on  the  lane  to  establish  coordinates  for  each 
spot. 


(2)  The  alarm  marker  poker  chips  are  surveyed  by  a  survey  team  using  DGPS  survey 
equipment  to  obtain  the  coordinate  array  (e.g.,  WGS  UTM84,  Northing  and  Easting)  for  each 
alarm  marker  (poker  chip)  position  (see  Figure  2).  The  surveyor  will  locate  each  poker  chip  on 
that  specific  lane,  identifying  it  by  the  number  on  the  chip.  After  the  completion  of  surveying  all 
the  placed  chips  of  a  given  mission  run,  that  surveyed  chip  count  is  compared  to  the  number  of 
marker  chips  dropped  by  the  data  collector  during  the  scanning  process.  This  second  quality 
check  verifies  all  chips  have  been  accounted  for  and  have  been  surveyed,  and  validates  the 
mission  run  by  the  test  officer  ensuring  a  QA  check  confirms  by  three  separate  sources:  data 
sheet  count,  actual  poker  chip  count,  and  number  of  surveyed  marker  chips.  If  all  three  confirm, 
that  mission  run  is  complete.  If  the  count  is  a  mismatch,  an  investigation  is  performed  by 
reviewing  data  sheet,  counting  poker  chip  alarm  markers,  and  reviewing  the  surveyed  ID 
numbers.  A  solution  is  concluded  prior  to  any  movement  of  chips. 

NOTE:  The  above  process  simplifies  the  mission  run  preparation,  scanning,  marking,  and 
identifying  alarm  markers  for  scoring.  It  does  not  address  any  operational  or  detection  issues 
during  mission  runs.  Scenarios  may  vary;  therefore,  standardized  corrective  action  procedures 
cannot  be  detennined  at  this  time.  The  hardware  TM/OM  must  be  reviewed  to  detennine  any 
actions  required  for  system  to  operate  proficiently;  if  no  corrective  actions  or  procedures  are 
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identified,  the  test  team  may  make  decisional  corrective  actions  to  get  system  to  an  operational 
status.  All  events,  malfunctions,  fixes,  solutions,  and  required  times  to  fix  issue,  is  documented 
in  test  logs  and  final  report. 


Figure  2.  Survey  of  alarm  marker  (poker  chip). 


(3)  Reconditioning  Test  Lanes  for  Continuous  System  Performance  Missions.  After 
all  alarm  markers  (e.g.,  chips)  and  survey  points  have  been  accounted  for,  the  alarm  marker 
poker  chips  can  be  removed  to  prepare  the  lanes  for  follow  on  mission  runs.  The  surveyed 
coordinates  are  forwarded  to  the  detection  scoring  team  for  performance  analysis.  Test 
personnel  will  perform  the  following  steps  in  preparation  for  subsequent  detection  runs  to 
maximize  the  validity  of  system  performance: 

(a)  Pick  up  and  arrange  the  numbered  alarm  markers  in  sequential  order. 

(b)  Back  drag  the  lane  (see  Figure  3)  removing  any  tracks,  marks,  footprints,  etc.,  on 
the  side  of  the  lane  to  ensure  there  are  no  visual  cues  left  behind  from  previous  scanning  process. 

e.  Performance  Data  Scoring  Methodology.  System  detection-performance  findings 
must  be  controlled.  Ground  truth  (target  positions)  must  be  treated  as  sensitive  information  and 
not  provided  to  system  representatives,  test  participants,  operators,  or  anyone  else  unless  there  is 
a  valid  need  to  know  (e.g.,  calibration  or  data  collection  specified  test  lanes,  scoring  analyst, 
etc.).  A  valid  need  to  know  is  determined  by  the  test  director,  project  manager,  or  independent 
evaluator.  Targets  will  be  emplaced  by  the  test  team  or  other  special  team  independent  of  the 
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test  participants.  The  initial  layout  of  each  test  lane/area  and  subsequent  changes  to  each  area 
will  be  controlled  by  the  test  director  and  documented  by  the  test  observers  (e.g.,  surveyors). 


6s;: 


m 


Figure  3.  Personnel  dragging  completed  lane  to  prepare  for  next  run. 


f.  The  primary  measures  of  performance  are  probability  of  detection  (percent  detection), 
faults  alann  rate,  and  advance  rate. 

(1)  Scoring  Definitions: 

(a)  Percent  Detection  (PD).  The  PD  provides  the  finding  of  target(s)  in  the  detection 
lane.  PD  is  determined  by  dividing  the  number  of  targets  successfully  detected  by  the  actual 
number  of  targets  encountered.  A  target  is  considered  detected  when  the  alarm  marker  (put  in 
place  by  the  operator  after  detector  indicates  a  detection  alarm)  is  emplaced  on  a  target  or  within 
a  target’s  halo.  A  halo  is  defined  as  the  area  within  any  external  dimension  of  the  target;  the  halo 
area  would  be  determined  by  the  scoring  analyst  or  evaluator.  It  is  possible  for  there  to  be  more 
than  one  accurate  detection  of  a  single  target  (redundant  detection).  For  the  purposes  of 
calculating  Percent  Detection,  only  one  detection  per  encounter  can  be  counted.  (NOTE:  even 
though  redundant  detections  are  not  considered  for  scoring,  they  are  also  not  classified  as  false 
alarms) 
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(b)  False  Alarm  Rate  (FAR).  The  FA  is  defined  as  alarm  marker  of  a  suspected  target 
that  falls  outside  the  halo.  The  FAR  typically  calculates  false  indications  by  the  detector  sensor 
over  an  area  (linear  meter  by,  usually,  1.5  meters  wide  lane  (wide  of  operator’s  detector  swath)). 
The  FAR  is  computed  as  the  number  of  false  alarms  divided  by  the  area  of  opportunity  for  false 
alarm  (scanned  area).  The  area  of  opportunity  for  false  alarm  is  computed  as  the  area  of  the  lane 
(or  sensor  path/detector  swath)  minus  the  area  of  the  targets  (and  clutter),  which  includes  the  area 
within  the  halo  radius. 

(c)  Advance  Rate  (AR).  The  AR  is  the  time  to  complete  a  section  of  lane/area  that 
you  are  scanning. 

(2)  The  detection  alarm  coordinates  will  be  provided  to  the  scoring  analyst  for 
processing  to  determine  detection  performance.  In  summary,  the  general  scoring  process 
concept  is  as  follows  by  the  scoring  team/analyst: 

(a)  Generate  plots  of  alann  marker  (e.g.,  poker  chip)  locations. 

(b)  Merge  alarm  plots  to  target  ground  truth  plots. 

(c)  Score  the  alarm  files  to  detennine  detection  performance  and  false  alarms.  This  is 
done  by  determining  the  standoff  distance  the  alann  mark  coordinate  point  is  from  the  target 
edge  (e.g.,  target  coordinates).  If  an  alarm  point  is  on  top  of  the  target  or  within  the  halo  buffer, 
it  is  scored  as  a  “detection”.  Any  distances  outside  the  halo  buffer  is  designated  a  “non¬ 
detection”  or  FA. 

NOTE:  Commercial  software  applications  are  typically  used  to  assist  the  scoring  analyst/team 
in  merging  the  digitized  alann  coordinates  with  the  digitized  target  ground  truth  coordinates  to 
provide  an  offset  distance  which  then  can  be  sorted  by  data  analysis  to  which  point(s)  are  inside 
and  outside  pre-set  distance  parameters  -  the  “halo”.  The  halo  distance  from  the  target  edge  can 
vary  (e.g.,  4-inches,  6-inches,  10-inches,  etc.)  depending  on  the  scoring  team,  evaluator,  or 
system  specification  scoring  criteria. 

(3)  Scoring.  For  a  target  detection  to  be  valid,  the  detection  alarm  location  must  fall 
within  the  “halo”  -  the  pre-determined  buffer  around  the  edges  of  a  target.  The  halo  edge  will 
follow  the  contour  of  the  target  edges  as  shown  in  the  halo  examples  in  Figure  4.  All  detection 
alarms  outside  the  halo  of  a  target  are  classified  as  FAs.  Detection  alarms  associated  with  clutter 
targets  in  the  lanes  are  scored  as  FAs.  For  a  valid  detection,  the  target  must  be  within  a  given 
halo. 
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Figure  4.  Examples  of  halo  boundary  configurations. 
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(a)  Detection  alarm  markers  may  be  placed  in  several  locations:  on  top  of  target  or 
within  the  halo,  outside  the  halo  or  away  from  a  target,  or  on  another  location  around  a  target 
(redundant  detection)  as  shown  in  Figure  5.  Redundant  detection  alarms  are  not  considered  and 
cannot  be  used  in  scoring.  Typically,  in  the  event  of  redundant  alarms,  the  closest  alarm  is 
declared  the  valid  detection  position.  Figure  6  is  a  lane  section  with  examples  of  detection  alarm 
possibilities.  Additional  scoring  analysis  procedures  provided  in  Appendix  G. 


Figure  5.  Example  of  detection  alarm  variations. 


Figure  6.  Lane  scoring  of  target  example. 

NOTE:  For  a  new  detection  system  with  unknown  performance  characteristics,  the  halo  size 
used  for  scoring  may  be  adjusted  (e.g.,  increased  or  decreased)  in  post  test  analysis  to  assess 
detection  performance  unidentified  characteristics. 
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(b)  Primary  scoring  formulas  for  Pd  and  FAR  are  shown  below: 


P 


D 


Number  of  Detections 
Number  of  Target  Encounters 


FAR  = 


Number  of  False  Alarms 
Lane  Area  -  Target  Area 


(c)  The  data  analyst  scores  the  detection  effectiveness  from  the  following  parameters: 

•  Calculates  the  Pd  for  each  target  type. 

•  Calculates  the  percent  detected  for  target  buried  depth. 

•  Calculates  the  percent  detected  for  each  sensor  mode  (metal,  GPR,  dual,  etc.). 

•  Calculates  the  percent  detected  for  entire  lane. 

•  Calculates  the  percent  detected  for  each  operator. 

•  Calculates  the  percent  detected  for  all  targets  as  a  whole. 

•  Calculates  False  Alarm  Rate. 

•  Calculates  Advance  Rate. 

NOTE:  Additional  detector  performance  analysis  can  be  perfonned  by  the  scoring  of  Pd  and 
FAR  using  Receiver  Operating  Characteristic  (ROC)  curves.  ROC  curves  are  used  to  assess  the 
performance  of  the  ground  penetrating  radar  detector.  ROC  curves  are  plots  of  the  Pd  vs.  the 
probability  of  false  alann  (Pfa)  for  a  given  signal-to-noise  ratio  (SNR).  Appendix  G  provides 
additional  narrative  description  for  scoring  and  ROC  curves. 

g.  Human  Factors  Assessment  Surveys.  The  test  schedule  should  allocate  time  for 
periodic  operator  (human  factors)  interviews,  questionnaires,  and/or  surveys  (e.g.,  post  training, 
mid-test,  and  post-testing).  Care  should  be  taken  to  ensure  that  the  test  participant  duty  day  is 
representative  of  the  mission  profile,  to  include  mission  gear  if  required  (e.g.,  helmet, 
ammunition  vests,  belts,  etc.).  An  example  operator  survey  is  included  in  Appendix  E. 

4.4  System  Safety  and  Human  Health. 

a.  Safety  observations  shall  be  made  throughout  all  testing;  therefore,  a  hazard  review 
shall  be  conducted  prior  to  any  testing  to  make  the  tester  aware  of  safety  measures.  Safety- 
specific  and  safety-related  tests  should  be  planned  in  addition  to  regular  operations,  to 
thoroughly  screen  the  SUT.  The  safety  testing  should  be  carried  out  early  in  the  test  cycle  to 
establish  and  confirm  inherent  safety  features. 
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b.  All  hazards  identified  should  be  evaluated  and  resolved  according  to  hazard  severity 
and  probability.  Mitigation  of  hazards  may  be  implemented  of  test  facility  to  protect  from  the 
fixed  design,  procedural  tasks,  or  focused  training. 

c.  Hazards  should  be  identified  and  characterized  as  to  severity  categories  and 
probability  levels  such  as  those  in  accordance  with  risk  assessment  levels  (U.S.  Army  Test  and 
Evaluation  (ATEC)  Regulation  (Reg)  385-1 5). 

d.  The  test  team  will  do  the  following  prior  to  and  during  testing: 

(1)  Prior  to  testing: 

(a)  Review  the  TM/OM,  prior  to  testing,  and  verify  that  actions,  identified  to  remove 
or  mitigate  recognized  hazards,  have  been  performed. 

(b)  Review  the  materiel  developer  or  contractor’s  Safety  and  Human  Health 
Assessment  Reports.  Coordinate  with  the  materiel  developer  to  ensure  a  RF  radiation,  laser,  or 
ionizing  radiation  study,  as  appropriate,  has  been  requested  and  performed,  or  completed  by  the 
U.S.  Army  Institute  of  Public  Health. 

(c)  Identify  safety  problems  during  test  and  their  classification  (hazard  severity  and 
probability  of  occurrence)  during  initial  inspections  and  during  testing. 

(d)  Complete  electrical,  mechanical,  or  miscellaneous  hazards  checklists  to  support  a 
safety  hazard  data  collection  plan. 

(e)  Document  discrepancy  findings  of  equipment  or  technical  publication  and 
training  material  review  (adequacy  of  warnings,  cautions,  and  notes). 

(f)  Review  MSDS  for  the  battery  and/or  any  other  hazardous  elements. 

(2)  During  testing: 

(a)  Record  photographic,  audiovisual,  or  other  documentation  of  hazards. 

(b)  Record  results  and  classification  of  toxic  fumes  (battery). 

(c)  Document  Human  Factors  and  Health  Hazards  issues:  whole  body  fatigue 
(weight  and  balance),  noise  measures  (alarms  and  headsets),  and  non-ionizing  or  ionizing 
radiation  levels. 

(d)  Assess  the  adequacy,  proper  functioning,  and  need  for  additional  or  improved 
safety  and  warning  devices;  i.e.,  guards,  interlocks,  alarms,  warning  lights. 

(e)  Provide  engineering  analysis  of  potential  for  and  documentation  of  any  safety 
issues/incidents. 
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(f)  Number  and  criticality  of  system-related  or  human/machine  interface  incidents. 

(g)  Document  operator  comments  and  recommendations  regarding: 

1  Adequacy  and  proper  functioning  of  safety  and  warning  devices  such  as 
guards,  interlocks,  or  alarms. 

2  Weight  and  balance  of  detector. 

3  Adequacy  of  safety  instructions,  guidance  in  user  manuals. 

4  Volume  and  intensity  of  audio  alert  tones. 

(3)  Post  testing: 

(a)  Complete  safety  report  identifying  safety  discrepancies,  test  incidents,  and 
information  with  respect  to  safety  issues  or  conditions. 

(b)  Provide  materiel  developer  and/or  contractor  findings  from  the  TM/OM 
review  and  testing  of  any  safety  and  operational  discrepancies  or  lack  of  information. 

4.5  Climatic  Suitability. 

a.  For  all  system  performance  trials,  meteorological  conditions  must  be  documented  and 
recorded  so  that  the  conditions  can  be  characterized  in  terms  of  system  assessment  in  reference 
to  climatic  conditions.  Perfonnance,  reliability,  discrepancies,  or  failures  can  be  quantified  as  a 
function  of  a  climatic  factor.  Other  types  of  climatic  suitability  problems  may  be  solved  through 
changes  to  design,  hardware,  procedure,  or  training.  Climatic  factors  should  encompass 
temperature,  solar  radiation,  precipitation,  wind,  salt  fog,  sand  and  dust,  pressure-altitude,  and 
immersion,  if  applicable  to  user  mission  requirements. 

b.  Testing  in  climatic  chambers  and  facilities  may  be  creatively  exploited,  desirably 
before  system  performance  trials,  if  facilities  can  permit  operation  of  detection  equipment. 
Where  desired,  screening  of  climatic  factors  should  be  combined  with  the  shock  and  vibration 
associated  with  transportation  and  handling  to  address  storage  and  transport  of  packaged 
equipment  prior  to  handling  and  operating.  Appendix  B  contains  those  test  standards  that  may 
be  used  if  comprehensive  screening  tests  are  required.  Induced  climatic  testing  may  not  be 
necessary  if  multiple  test  sites  are  selected  that  provide  a  range  of  climatic  conditions  in  which 
performance  trials  that  could  include  the  complete  life  cycle  assessment  (e.g.,  storage, 
operational  use,  maintenance,  etc)  in  accordance  with  the  requirements  document  and  associated 
operational  mode  summary/mission  profile.  See  MIL-STD-8 10G  for  additional  detailed 
guidance  on  climatic  suitability  testing. 

4.5.1  Natural  Environment  Performance  Testing. 
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a.  The  perfonnance  of  the  detection  systems  will  depend  very  much  on  the  environment 
to  which  the  system  is  being  operated  in.  Soil  condition,  natural  and  artificial  (composition  and 
moisture  content),  diurnal  and  seasonal  differences,  climate,  vegetation,  and  terrain  are  a  few 
considerations. 

b.  The  use  of  natural  environments  at  test  ranges  and  climatic  selected  test  areas  are  more 
desirable  than  using  test  chambers.  Based  upon  definition  of  natural  extreme  climatic 
conditions,  conditions  expected  to  be  encountered  in  storage,  transit,  transport,  and  operational 
should  be  screened  using  single  or  multiple  factor  test.  Climatic  factors  (see  paragraph  4.5.2) 
should  encompass  temperature,  solar  radiation,  sand  and  dust,  rain,  salt  fog,  etc.  Scheduling  tests 
in  natural  environments  should  be  planned  during  winter,  summer,  dry,  and  wet  extreme 
conditions  for  the  area  selected  to  test  items.  In  accordance  with  MIL  STD-8 10G,  the  general 
guidance  on  appropriate  suitability  for  testing,  appropriate  for  the  intended  application,  and  the 
level  of  item  assembly  required  for  test  should  be  tailored  to  simulate  operational  mission  modes. 
In  general,  the  SUT  should  be  tested  at  the  system  level.  Test  facility  induced  instrumented 
systems  or  subcomponents  should  be  used  to  enhance  data  acquisition  quality  assurance, 
inspections,  and  failure  analysis  or  diagnostics,  where  it  can  be  shown  that  results  are  not 
compromised  (e.g.,  interoperability)  or  that  results  are  better  quantified.  Care  should  be  taken  to 
ensure  that  integral  instrumentation  is  sufficiently  rugged  to  withstand  natural  and  induced 
environments  subjected  during  testing.  Some  example  testing  considerations  that  should  be 
addressed  and  considered  to  assess  climatic  operational  mission  requirements  include: 

(1)  Determine  testing  procedure  method(s)  (e.g.,  storage,  operational,  manipulation, 
exaggerated,  etc.). 

(2)  Exposure  conditions:  exposed  or  sheltered. 

(3)  Exposure  duration. 

(4)  Exposure  temperature  at  a  constant  state  or  cyclic/diurnal  cycling. 

(5)  Packaging  configuration  (e.g.,  shipping  or  storage  containers,  bare,  single  item  or 
palletized  configurations,  etc.). 

(6)  Representative  climate  category. 

(7)  Spectral  distribution:  sea  level  versus  high  ground  elevations. 

(8)  Airflow,  if  applicable. 

(9)  Operational  checkout  points  (e.g.,  pre-checks,  periodic  throughout  testing,  post¬ 
check,  etc.). 

c.  Ambient  Air,  Outdoor  Tests.  Conduct  tests  of  this  type  under  the  extreme  natural 
conditions  available.  Tests  will  be  conducted  under  full  sun  and  preferably  when  the  air 
temperature  is  at  an  extreme,  consistent  level.  Additional  subtest  assessments  may  include: 
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(1)  Human  factors. 

(2)  Exercising  all  components  and  ancillary  equipment. 

(3)  Checkout  of  all  electrically  operated  equipment  and  lights. 

(4)  Power  consumption. 

(a)  Low-Temperature  Tests  (Operational  and/or  Storage).  The  low-temperature 
requirements  should  be  found  in  the  requirement  documents  or  the  equipment  specifications. 

The  requirements  for  low-temperature  performance  are  not  consistent  for  all  systems  (typically 
because  of  use  mission  requirements);  the  temperature  at  which  certain  systems  may  be 
introduced  often  varies.  The  guidance  document  must  be  examined  to  detennine  the  exact  cold- 
weather  performance  requirements  for  the  system  under  test.  For  storage  type  tests, 
thennocouples  may  be  installed  at  all  points  where  temperature  monitoring  can  be  performed. 

(b)  High-Temperature  Tests  (Operational  and/or  Storage).  The  high- temperature 
requirements  should  be  found  in  the  requirement  documents  or  the  equipment  specification.  For 
storage  type  tests,  thermocouples  may  be  installed  at  all  points  where  temperature  monitoring 
can  be  performed.  Ambient  high  temperature  testing  should  be  above  32  “Celsius  (C) 

(90  “Fahrenheit  (F)). 

d.  Other  Conditions  (to  assess  operational  and  human  factors  elements)  include: 

(1)  Assess  the  SUT  to  operate  in  various  environmental  climates  (e.g.,  hot  desert, 
tropical,  cold  (frozen)  tundra,  and  surf  (beach). 

(2)  Assess  in  direct  sunlight,  and  overcast  cloud  cover. 

(3)  Assess  in  windy,  blowing  conditions  (e.g.,  dusty,  blowing  snow,  etc.). 

4.5.2  Induced  or  Chamber  Enviromnental  Conditioning  Testing. 

Where  natural  climatic  temperature  testing  cannot  be  perfonned,  simulated  temperature 
conditioning  may  be  required  to  assess  the  SUT  in  both  the  storage  and  operational 
configurations. 

a.  Testing  in  climatic  chambers  and  facilities  could  be  creatively  explored;  however,  the 
size  and  scope  of  testing  desired  (e.g.,  detection  assessment  over  targets)  could  severely  restrict 
the  use  of  environmental  chambers  other  than  basic  operational  performance  parameters,  (e.g., 
on/off,  power  limitations,  operator/machine  interface,  electronic  parameters,  etc.). 

b.  The  effect  of  an  environment  on  the  aspect  of  system  safety  should  also  be  tested. 
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c.  The  minimum  sample  size  for  any  exposure  should  be  detennined  by  the  configuration 
of  the  system  being  tested.  Any  existing  data  from  the  developer/contractor’s  may  be  reviewed. 
The  developer's/contractor’s  tests  can  be  monitored  and/or  witnessed,  and  a  determination  made 
as  to  whether  to  accept  the  data  or  recommend  additional  screening  tests. 

4.5.3  High  Humidity. 

The  SUT  may  need  to  be  tested  in  high  humidity  environments  to  determine  the  affects  of 
corrosion,  electrical  malfunctions,  swelling  of  nonmetallic  materials,  loss  of  insulating  qualities, 
deterioration  of  hygroscopic  materials,  etc.  If  systems  have  an  LED  screen,  fogging  or  moisture 
accumulation  may  be  observed. 

4.5.4  Fungus  Resistance. 

Fungus  resistance  can  be  ascertained  by  an  examination  of  the  materials  composition  of  the  SUT, 
and  from  certification  by  the  developer  that  the  materials  used  in  the  test  item  are  fungus-inert  or 
impregnated  with  fungus-resistant  material.  If  considered  necessary,  specific  fungus  tests  should 
be  planned.  Allow  a  minimum  of  twenty  eight  days  for  fungus  germination  and  exposure 
analysis. 

4.5.5  Salt  Spray  (Fog). 

A  salt  spray  (or  salt  fog)  test  may  be  necessary  if  the  SUT  is  likely  to  be  operated  or  deployed  in 
a  marine  environment,  to  include  surf  beach  regions. 

4.5.6  Sand  and  Dust. 

This  test  is  conducted  when  it  is  possible  that  sand  or  dust  could  interfere  with  moving  parts, 
fans,  hand  controls,  etc.  One  or  multiple  testing  procedures  should  be  selected;  testing 
procedures  consist  of  Procedure  1  -  Blowing  Dust;  Procedure  2  -  Blowing  Sand;  and  Procedure  3 
-  Settling  Dust. 

4.5.7  Solar  Radiation. 

This  test  is  primarily  conducted  for  heat  effects.  In  view  of  the  previous  high  temperature  tests, 
the  solar  radiation  test  may  be  unnecessary.  One  or  multiple  testing  procedures  should  be 
selected;  testing  procedures  consist  of  Procedure  1  -  Cycling  Heating  Effects;  and  Procedure  2  - 
Steady  State  Actinic  Effects. 

4.5.8  Immersion. 

Depending  upon  the  requirements,  an  immersion  (water-tightness)  test  should  be  carried  out  in 
addition  to  the  rain  test  to  check  seals  and  protection  of  electrical  connections. 
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4.5.9  Rain  and  Freezing  Rain. 

An  immersion  test  may  make  a  rain  test  unnecessary.  Freezing  rain  tests  should  also  be 
considered  in  conjunction  with  the  low  temperature  tests.  One  or  multiple  testing  procedures 
should  be  selected.  Testing  procedures  consist  of  for  the  following: 

a.  Rain. 

(1)  Procedure  1  -  Rain  and  Blowing  Rain. 

(2)  Procedure  2  -  Exaggerated. 

(3)  Procedure  3  -  Drip. 

b.  Freezing  Rain. 

4.6  Transportation  and  Handling. 

a.  Transportation  and  handling  tests  will  reflect  the  logistics  and  mission  environment. 
Specific  screening  tests  will  be  conducted  as  a  necessary  measure  to  support  transportability 
certifications  and  safety  assessments.  Screening  tests  will  focus  on  intermodal  transport,  air 
delivery  (containerized),  and  handling  related  shock  and  vibration  effects  on  the  item  and  will  be 
coupled  with  climatic  conditions.  Detailed  technical  inspections  will  be  incorporated  into  all 
tests  involving  shock  or  vibration  whether  they  are  conducted  singly  or  in  sequence  as  a  test 
control.  Packaging  configuration  should  be  tailored  to  the  planned  life  cycle  (palletized,  crated, 
bare).  Testing  should  be  planned  to  explore  whether  the  item  is  safe  to  handle  and  operate 
following  transportation  and  handling  subtests;  these  samples  could  possibly  also  be  used  as  part 
of  the  reliability,  or  perfonnance  subtests,  as  well  as,  safety  data  to  support  safety  assessment  of 
SUT  hardware.  If  necessary,  special  tactical  mockups  may  be  fabricated  to  physically  simulate 
the  carrier  (vehicle  bed,  helicopter  storage  compartment,  storage  rack,  etc.). 

b.  A  typical  logistical  depot/mission/field  movement  profile  should  be  constructed  to 
identify  transportation  and  handling  subtest  specifications  (e.g.,  port  or  air  transport  staging  area 
-  transport  to  forward  supply  point  (by  truck  or  helicopter)  -  then  transported  and  handled  at 
mission/field  use  point  (by  personnel  carriers,  Mine-Resistant  Ambush  Protected  (MRAP) 
vehicles,  dismounted,  etc.)).  Tactical  vibration  profiles  should  be  developed  or  identified,  and 
used  for  all  items  to  simulate  being  transported  in  through  the  logistics  life  cycle.  These 
vibration  tests  could  be  in  conjunction  with  other  logistics-related  sequential  testing  (e.g., 
secured  cargo  (container/palletized),  loose  cargo  (container,  bare,  or  both),  and  7-ft  drop 
(packaged  in  container  or  other  carrying  apparatus),  and  4-ft  drop  (bare)  subtests). 

c.  Appendix  B,  TableB-1  contains  those  test  standards  that  may  be  used  if  screening  tests 
are  required.  Screening  tests  will  focus  on  ground,  water,  and/or  air  transport  (containerized),  air 
delivery,  and  handling  related  shock  and  vibration  effects  on  the  item  and  should  be  coupled  with 
climatic  conditions  as  stated  above.  Other  testing  sources  are  outlined  in  the  following: 
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STANAG  4138  (Vibration  Resistant  Equipment)6,  and  MIL-STD-810G  (Method  514  - 
Vibration). 

NOTE:  Special  considerations  are  testing  with  or  without  power  supply  (e.g.,  battery(s),  etc.), 
and  at  what  intervals  to  perform  an  operational  check  using  standardized  target  sample(s)  for 
each  sensor  component. 

4.7  Integrated  Logistics  Supportability. 

a.  Logistics  Supportability  does  not  necessarily  lend  itself  to  standardized  test  design  in  a 
technical  testing  forum.  Therefore,  overview  assessments  for  logistic  supportability  can  be 
integrated  with  Logistic  Support  (LS)  requirements  (packaging),  RAM,  and  the  Human  Factors 
data  elements  of  the  SUT.  Data  are  collected  during  all  phases  of  testing.  Suggested  focus 
categories  for  assessment  that  should  be  relative  to  requirements,  observations  and  consideration 
are  outlined  as  follows: 

(1)  Transportability. 

(2)  Equipment  publications. 

(3)  Training. 

(4)  Test,  measurement,  and  diagnostic  equipment. 

(5)  Tools. 

(6)  Spares  and  repair  parts,  etc.. 

b.  The  test  team  should  perform  the  following: 

(1)  Assess  the  shipping  container  or  packaging  configuration  for  the  following: 

(a)  Protection  of  equipment  and  accessories. 

(b)  Adequacy  of  securing  all  components. 

(c)  Ease  of  use  and  handling  (e.g.,  opening  and  closing,  extracting  and  stowing 
of  components,  etc). 

(d)  Ability  to  stack;  maximum  quantities  allowable  for  stacking. 

(e)  Size  and  weight. 
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c.  Assess  the  TM/OM  for  completeness,  comprehension,  troubleshooting 
procedures  and  cautions/warnings.  Discrepancies  should  be  identified  to  assist  in  the 
updating  of  the  TM/OM  for  future  updates. 

NOTE:  For  any  TM/OM  safety  discrepancies,  mitigating  measures  should  be 
implemented  to  protect  operator(s)  and/or  test  personnel  during  test  and  evaluation 
assessment  trials. 

d.  Review  any  proposed  training  plans  needed  for  test  personnel  to  perform  the 
test  and  evaluation  assessment  of  the  SUT.  Some  training  requirements  for  testers  may  be 
modified,  shortened,  and  enhanced  than  what  is  developed  for  military  user  operators. 

However,  this  training  may  provide  the  Program  Manager  (PM),  military  trainer,  and/or 
contractor  developer  opportunities  to  detennine  what  are  the  important  and  effective 
points/practices  to  incorporate  into  the  overall  military  user  formal  training  program. 

e.  Inventory  any  tools  and/or  special  pieces  included  with  the  SUT  that  are 
required  for  use  and/or  for  operator  level  maintenance.  Identify  any  tools  required  not 
provided  or  listed  in  TM/OM  that  were  necessary  to  operate,  maintain,  or  repair  SUT.  If 
any  unique  or  measuring/diagnostic  equipment  is  necessary  to  operation,  maintain,  or 
repair  the  SUT  not  listed  in  TM/OM,  indicate  as  a  discrepancy. 

f.  Inventory  all  spare  parts  included  with  each  SUT.  Document  any  spare  parts 
supplied  during  testing.  Determine  if  these  spare  parts  are  commercial  off  the  shelf  items 
or  specialized  parts/components.  Having  backup  SUT  or  any  specialized 
parts/components  on  hand  during  testing,  may  prevent  or  reduce  significant  down  time 
during  testing  if  a  failure  is  experienced. 

g.  Other  elements  for  assessment  considerations: 

(1)  Adequacy  and/or  availability  of  special  tools,  test,  measurement,  or 
diagnostic  equipment. 

(2)  Availability  of  batteries  (e.g.,  supply,  commercial,  commonality,  etc.). 

4.8  Reliability,  Availability,  and  Maintainability  (RAM). 

While  testing,  collect  operational  and  maintenance  data  to  determine  overall  ability  of  the  system 
to  complete  task/mission.  Record  RAM  data  elements  to  detennine  mean  time  between 
maintenance  ratios,  operational  time  between  battery  replacement,  etc.  Track  scheduled  and 
unscheduled  maintenance  tasks,  as  well  as,  preventative  maintenance  procedures.  Testing  should 
be  planned  to  examine  reliability  and  durability  of  all  SUT  essential  features  with  special 
emphasis  on  the  reliability  of  safety  features. 

NOTE:  Record  any  compatibility  problems  noted  during  operations  (human  factors)  or 
maintenance. 
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4.8.1  Reliability. 

The  reliability  testing  of  the  SUT  requires  the  conduct  of  a  significant  number  of  repeated 
activities  and  operational  run  time.  The  activities  will  be  based  on  a  typical  operational  mission 
of  the  SUT  and  should  be  defined  in  the  user  requirement  documents.  The  number  of  repeats 
(e.g.,  target  encounters,  operational  time,  etc.)  required  will  be  established  to  ensure  statistical 
validity  of  the  results. 

a.  Perform  daily  operator  PMCS  as  specified  in  the  TM/OM. 

b.  During  system  testing,  collect  data  on  the  battery  usage,  record  average  battery  life, 
and  adequacy  of  low  battery  indicators  prior  to  system  degradation/shutdown.  Include  data  on 
ease  of  battery  replacement.  Also  note  the  availability  of  the  required  battery  (rechargeable  or 
not)  and  whether  they  are  commercial-off-the-shelf  or  military  type  batteries.  Key  battery  data 
elements  to  monitor: 

( 1 )  Battery  operational  time . 

(2)  Battery  charge  time. 

(3)  Adequacy  of  battery  charger. 

c.  In  the  event  of  a  system,  subsystem,  or  component  failure,  malfunction,  or 
performance  inadequacy,  collect  data  on  faults,  discrepancies,  and  issues,  and  record  in  test  log 
books. 


(1)  Description  of  failure,  malfunction,  or  performance  inadequacy  causing  a  system 
performance  discrepancy. 

(2)  Description  and  function  of  the  specific  subsystem  or  component  that  failed  or 
having  questionable  operational  issues. 

(3)  Description  of  subsystem  or  component  discrepancy,  if  possible,  and  impact  on 
system  perfonnance. 

(4)  Description  of  corrective  action/troubleshooting  procedures,  including  time  and 
level  of  maintenance  required  to  correct  discrepancy. 

(5)  System  perfonnance  after  corrective  action. 

(6)  Repairs  and/or  maintenance  beyond  the  scope  outlined  the  TM/OM  performed  by 
the  Field  Service  Representative  or  test  personnel  (with  or  without  developer’s  guides). 
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4.8.2  Maintainability. 

a.  All  maintenance  operations  required  during  the  course  of  testing  should  be  performed 
in  accordance  with  the  applicable  TM/OM,  or  recorded  if  the  system’s  Field  Service 
Representative  performs  the  maintenance  task.  Preventive  maintenance  should  be  performed  at 
the  specified  intervals.  All  crew,  organizational,  direct  and  general  support  maintenance  actions 
should  be  monitored  to  accumulate  required  data.  Operator's  daily  checks  and  services  should  be 
observed  sufficiently  to  obtain  representative  times  for  perfonning  these  tasks.  A  database 
should  be  used  in  the  collection  and  analysis  of  this  logistic  supportability  data  (e.g.,  test  incident 
reports  (TIRs)  should  be  posted  in  the  Army  Test  Incident  Reporting  System  (ATIRS)). 

b.  No  simulated  maintenance  should  be  perfonned.  All  maintenance  operations  required 
to  maintain  the  system  in  an  operational  status  should  be  monitored.  Factors  which  establish  that 
ease  of  maintenance  should  be  examined  relative  to  the  design  of  the  system.  Assess  and 
document  the  reliability  of  components  and  other  factors  which  indicate  that  the  equipment 
design  either  has  or  has  not  been  directed  toward  minimizing  maintenance.  TM/OM  should  be 
used  as  a  guide  in  determining  maintainability-design  characteristics.  If  none  are  provided, 
request  instructional  guides  from  the  field  service  representative  or  system  developers,  and 
record  executed  maintenance  procedures  utilized  for  a  given  maintenance  task.  These 
procedures  can  then  be  provided  to  system  developer  as  a  template  to  start  a  guide  or  manual. 
Maintenance  checklists,  observational  data  records,  and  failure  reports  are  used  to  acquire  the 
maintainability  data. 

c.  Throughout  all  maintenance  operations,  observations  involving  the  man-item 
relationship  and  safety  of  maintenance  should  be  recorded. 

d.  During  all  preventive  and  corrective  maintenance  tasks  performed,  the  appropriate 
manuals  provided  for  support  of  the  test  items  should  be  reviewed,  and  comments  on  their 
accuracy,  adequacy,  and  safety  instructions  for  personnel  and  equipment,  to  include 
environmental  protection  during  operation  and  maintenance  (e.g.,  battery  disposal),  should  be 
identified  and  recorded. 

e.  Repair  parts  should  be  examined  with  respect  to  the  maintenance  category  authorized 
to  stock  and/or  requisition  the  part,  and  the  category  prescribed  to  replace  the  part.  Parts  should 
be  examined  to  ensure  modular  design  has  been  considered  and  they  should  be  compared  with 
the  repair  parts  outlined  in  the  TM/OM;  this  assesses  the  TM/OM  to  ensure  it  is  adequate. 

f.  During  all  maintenance  operations,  the  tools  and  test  equipment  authorized  in  the 
maintenance  support  literature  should  be  used  in  accordance  with  instructions  contained  in 
applicable  equipment  publications.  If  such  a  list  of  authorized  tools  and  test  equipment  is  not 
provided,  verify  with  system  developer  that  suggested  tools/test  equipment  will  work  for 
required  maintenance,  and  seek  their  authorized  for  use  in  the  maintenance  of  the  SUT. 
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4.9  Human  Factors  or  Ergonomics. 

Human  factors  assessment  needs  to  be  integrated  into  all  phases  of  testing.  Quantitative  and 
qualitative  data  are  gathered,  as  applicable,  for  both  the  packaged  and  bare  items  relative  to 
functions  such  as  assembly/disassembly,  operability,  transportability,  portability,  and  usability. 
Specific  human  factors  test  procedures,  checklists,  and  questionnaires  will  be  used  (see 
Appendix  E  for  examples)  to  examine  design,  tasks,  operator  performance,  and  adverse  natural 
or  induced  environmental  conditions. 

a.  Standardized  and/or  customized  questionnaires  are  administered  to  test  team 
participants  (operators)  during  and  after  system  performance  trials  to  aid  in  appraisal  of  training, 
hardware,  and  equipment  publications.  Questions  and  request  for  comments  should  direct 
operator  responses  to  provide  more  than  a  “Yes/No”  reply.  Encourage  descriptive  responses  to 
provide  liberal  feedback,  both  constructive  and  negative,  even  suggesting  their  recommendations 
or  ideas. 

b.  Adverse  environmental  conditions  should  be  included  to  examine  the  effects  of 
extreme  climates  and  other  induced  conditions  such  as  nuclear,  biological,  chemical  (NBC) 
relative  to  compatibility  with  combat  clothing  and  equipment  on  the  ability  to  perfonn  key 
functions  (e.g.,  system  assembly,  scanning  operations,  change  out  of  batteries,  monitoring 
alarms,  etc.).  Test  design  should  include  simulation  of  mission  or  work  cycles.  Personnel 
should  include  trained  engineering  technicians,  as  well  as,  personnel  who  are  representative  of 
the  intended  military  user  population  in  terms  of  skills,  size,  strength,  and  wearing  suitable 
gannents  and  equipment  appropriate  to  the  tasks.  Data  will  be  gathered  to  identify  and  define 
the  test  participants  (user  population).  Sufficient  data  will  be  acquired  to  establish  the 
demographics,  anthropometry,  skills,  grades,  experience,  gender,  handedness,  and  sensory  acuity 
(visual  and  auditory).  Subjective  data,  including  that  from  interviews  and  questionnaires,  should 
be  taken  during  testing  and  should  be  repeated  to  show  learning  effects  as  well  as  to  examine  the 
ease  of  use.  Interviews  should  be  structured  and  surveys  scaled  to  provide  quantitative 
comparison  of  responses. 

NOTE:  Minimize  the  collection  personal  identification  infonnation  (PII)  and  prevent  storage  of 
PII.  Code  the  operators  (e.g.,  Operator  A,  Operator  B,  etc.),  and  correlate  the  demographics, 
anthropometry,  and  other  information  to  the  operator’s  code  ID. 

NOTE:  If  key  user  representatives  are  not  available,  a  complete  training  program  needs  to  be 
provided  to  the  selected  operators  so  they  can  become  confident  to  operate  the  SUT  and  to  at 
least  provide  applicable/knowledgeable  comments  on  the  SUT  in  questionnaires  and/or 
operational  surveys. 

c.  Additional  human  factors  engineering  data  may  be  gathered  to  examine  operator 
health  environments  to  ascertain  the  severity  of  hazards  associated  with  use.  These  may  include 
non-ionizing  radiation  hazards  due  to  lasers  or  radio  frequency  communication,  impulse  noise 
and  overpressure,  weight,  and  sharp  edges.  Findings  that  indicate  hazard  severities  that  require 
mitigation  will  be  collated  to  the  safety  subtest.  The  best  way  to  measure  the  audible  warning 
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sound  piped  into  a  headset  is  by  using  specialized  manikins,  following  American  National 
Standards  Institute  (ANSI)  S12.427  procedures. 

d.  An  analysis  of  the  SUT  may  be  required  to  identify  the  requirements  for  operator  and 
maintenance  training.  Other  agencies  may  have  input  for  training  and  maintenance 
requirements,  and  these  procedures/processes  may  be  integrated  during  testing  to  review  the 
practicability  of  any  identified  implemented  procedure/process. 

4.9.1  Human  Factors  Testing  Elements  for  Considerations. 

a.  Record  demographics  and  anthropometry  elements  of  test  participants,  to  include 
corresponding  apparel  (Mission  Oriented  Protective  Posture  (MOPP),  arctic,  battle  dress 
uniform)  and  accessories  (pack  frame,  weapon,  etc.)  on  test  participant.  Data  should  include 
MOS  or  civilian  equivalent,  time  in  specialty,  and  years  and  months  of  related  equipment  use. 

b.  Provide  operator  and  maintainer  questionnaires  to  complete. 

c.  Record  observation  assessments  of  operator(s)  during  operational  use. 

d.  Assess  hardware  design: 

(1)  Lighting  measurements  of  displays  -  night  and  day. 

(2)  Noise  levels  related  to  operational  awareness  and  detection  alarms. 

(3)  Labeling  and  color  coding/marking  of  safety  warnings  and  placement  locations. 

(4)  Weight  of  test  item  in  packaged  and  unpackaged  state,  handholds,  latches,  etc. 

(5)  Ease  and  comfort  while  holding  detector  for  detection  scanning. 

(6)  Balance  of  system  to  assist  with  operator  performance  of  scanning  duties. 

e.  Assess  the  portability  and  human  interface  features  (e.g.,  easy  of  comfort,  hand-grip 
ergonomics,  operational  stress  points  on  operator  associated  with  straps,  balance,  rubbing  against 
body,  heat  elements,  etc.). 

f.  Determine  the  capability  of  mine  alarm  signals  to  inform  operator: 

(1)  Description  of  the  types  and  characteristics  of  signals. 

(2)  Description  of  any  intensity  adjustability  of  signals. 

(3)  Adequacy  of  the  ease  of  discriminating  visual  display  signals  from  other 
operational  indicators. 
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(4)  Adequacy  of  the  audio  signal  intensity  in  typical  operational  environments. 

(5)  Operator  comments  on  the  ease  of  interpreting  the  meaning  of  visual  and  audio 

signals. 

g.  Assess  the  attachability  (if  applicable)  of  SUT  hardware  to  operator  tactical  belts, 
vests,  or  other  accessories  (e.g.,  battery  packs,  carry  straps,  spare  parts,  etc.): 

(1)  Ratings  of  ease  of  attaching  system  components  to,  and  removing  components 
from,  human  accessory. 

(2)  Documented  observations  of  any  difficulties  with  lifting,  positioning,  and 
securing  components  on  operator. 

(3)  When  SUT  is  attached  to  operator,  description  of  any  operator  limited  or 
restricted  movement. 

h.  Document  the  ability  of  the  operator  to  perform  PMCS  in  the  required  time  using 
instructions  provided  with  the  system  hardware. 

(1)  Ease  of  performing  PMCS. 

(2)  Time  to  perform  PMCS  during  the  day  and  night. 

(3)  Adequacy  of  instructions. 

(4)  Ability  of  operators  to  operate  the  system  while  attired  in  environmentally 
protective  clothing. 

(5)  Listing  of  all  protective  clothing  ensemble  components  worn  (e.g.,  civilian  street 
attire,  military  battle  duty  unifonn,  coverall/overalls,  cold  weather  gear,  biological  attire,  etc.). 
Also  identify  if  hearing  audio  ear  piece  restricts  operator  to  wear  head  gear  or  helmet. 

(6)  Compatibility  of  SUT  hardware  with  protective  clothing,  with  an  emphasis  on 
hand  gear  (e.g.,  gloves). 

(7)  Documented  observations  of  task  completion  difficulties  (e.g., 
assembly/disassembly,  operating,  PMCS,  battery  replacement,  etc.)  experienced  by  operators. 

(8)  Ease  or  difficulty  of  adjusting  controls  barehanded,  wearing  protective  gloves, 
and  while  wearing  NBC  and  cold  weather  protective  gloves. 

i.  Document  the  ease  or  difficulty  of  the  ability  of  the  operators  to  set  up,  operate,  and 
maintain  the  system: 
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(1)  Adequacy  of  system  to  alert  the  operator  to  buried  threats  (e.g.,  mines,  IEDs, 
pressure  plates,  IED  wire,  etc.). 

(2)  Adequacy  of  hardware  to  verify  locations  of  buried  threats  (e.g.,  mines,  IEDs, 
pressure  plates,  IED  wire,  etc.). 

(3)  User  confidence  regarding  the  ability  of  system  to  accurately  find  and  pinpoint 
buried  threats  (e.g.,  mines,  IEDs,  pressure  plates,  IED  wire,  etc.). 

(4)  Description  of  operator  errors,  as  well  as,  their  consequences  on  overall  system 
performance  (e.g.,  in  correct  calibration,  power  level  awareness,  continuous  scanning  height 
above  ground,  etc.). 

(5)  Adequacy  of  progressing  through  SUT  menu  during  PMCS,  pre-operation 
calibration,  during  detection  scanning  operations,  warning  resets,  error  resets,  etc. 

j .  Illustrative  photographs  when  measurements  or  user  comments  show  a  problem  that 
reflects  a  human  machine  interface  issue. 

k.  Observe  operators  if  mission  fatigue  causes  operational  issues,  but  not  limited  to: 

(1)  Scanning  technique  becoming  erratic  (e.g.,  not  scanning  the  ground  evenly  across 
the  surface  at  the  specified  height  and  speed,  etc.). 

(2)  Fatigue  causing  the  operator  to  shorten  the  scanning  duration,  requiring  more 
frequent  breaks. 

4.10  Electromagnetic  Environmental  Effects  (E3)  and  Vulnerability. 

a.  Testing  should  be  designed  to  reveal  vulnerabilities  of  design  or  hardness  of  the  SUT 
design.  Vulnerabilities  could  be  false  or  improper  detections  or  reversible/irreversible 
degradation  following  exposure  to  the  phenomenon  (e.g.,  simulated  E3  testing  effects).  Where 
possible,  testing  that  reveals  vulnerabilities  should  also  examine  whether  hardening, 
proliferation,  or  dispersion  would  solve  the  specific  vulnerability. 

b.  Perform  testing  to  the  degree  necessary  to  determine  the  effects  of  E3  upon  the 
operational  capability  of  the  SUT  with  respect  to  its  operational  role  in  the  military  forces, 
equipment,  and  systems.  E3  testing  should  encompass  all  electromagnetic  disciplines,  including 
electromagnetic  compatibility/electromagnetic  interference,  electromagnetic  radiated  emissions, 
electromagnetic  vulnerability,  electronic  pulse,  electronic  counter-countenneasures,  hazards  of 
electromagnetic  radiation  (EMR)  to  personnel,  ordnance  and  volatile  materials,  and  natural 
phenomena  effects  of  lightning  and  electrostatic  discharge.  Levels  of  E3  phenomenology  shall 
be  based  on  projections  of  threat  and  friendly  offensive  radio  frequency  (RF)  capabilities,  tactical 
and  fixed  radars,  tactical  and  fixed  communications  and  electronics,  commercial  emitters, 
broadcast  stations,  amateur  radio  services,  and  threat  and  friendly  jammers.  E3  testing  is 
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considered  part  of  vulnerability  testing,  but  is  given  special  consideration  due  to  the  prevalence 
of  E3  in  the  logistics  environment  and  on  the  modern  battlefield. 

c.  If  E3  testing  is  deemed  necessary,  specialized  test  equipment  may  be  used  to  provide 
controlled  emissions  for  testing.  The  following  procedures  can  be  used  as  a  conceptual  guide. 

(1)  Place  the  SUT  antenna  above  ground  about  the  height  used  to  scan  the  ground  in 
an  operational  mode.  Have  a  calibration  target  place  to  perform  pre-emitting  scans  and  to  scan 
during  test  emissions. 

(2)  In  accordance  with  testing  standards  the  RF  environment  should  be  adjusted  to 
generate  a  continuous  wave  (CW)  and  pulse  modulation  field  at  the  desired  frequency.  The 
operational  environment  should  be  adjusted  to  generate  amplitude  and  frequency  modulation 
fields. 


(3)  Transmitter  power  should  be  slowly  increased  until  either  the  criterion  field  or 
facility  limitation  or  personal  exposure  limits  (PEL)  are  reached. 

d.  The  above  procedure  should  be  repeated  for  all  selected  frequencies,  modulations, 
polarizations,  and  orientations. 

e.  For  observed  anomaly,  the  threshold  field  intensity  at  which  the  malfunctions 
appeared  should  be  determined  by  additional  test  runs  in  which  the  E3  test  environment  is 
increased  until  the  field  intensity  is  reached  at  which  the  anomaly  occurred.  This  additional  test 
run  will  also  suffice  as  a  check  to  ensure  that  the  electro-interference  was  the  cause  of  the 
anomaly. 

f.  For  compatibility  testing  of  RF  emissions  (e.g.,  jammers,  tactical  radios,  etc.), 
determine  what  the  operational  distance  can  be  performed  without  degradation  to  the  detection 
system  normal  operating  modes.  With  friendly  emitters  off,  scan  over  a  valid  target;  with 
emitters  on,  scan  over  target  to  detennine  if  any  degradation  is  experienced.  Repeat  by 
expanding  the  separation  distance  between  the  detector  and  emitter  until  no  degradation 
observed. 

NOTE:  Testing  results  typically  are  classified  and  data  security  handling  measures  must  be 
considered. 

4.11  Software. 

a.  Software  testing  techniques  will  vary  dramatically  and  may  not  lend  themselves  to 
standardization.  Software  should  be  baselined  prior  to  test  start  and  should  be  maintained 
throughout  each  test  session.  See  ITOP  04-2-5208  for  guidance  on  software  testing.  Testing 
software  is  necessary  to  ensure  that  safety  is  designed  into  the  software  algorithm,  and  that  safety 
is  maintained  throughout  the  software  life  cycle.  It  provides  uniform  procedures  for  developing 
and  implementing  a  safety  critical  software  test  methodology  to  identify  the  software  caused 
hazards/faults.  The  objective  is  to  ensure  that  the  software  design  takes  positive  measures  to 
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enhance  system  safety,  and  that  software  errors  which  could  reduce  system  safety  have  been 
eliminated  or  controlled  to  an  acceptable  level  of  risk. 

b.  The  test  team  will  capture  following  data  elements: 

(1)  Software  version  at  start  of  system  performance  trials. 

(2)  Problems  attributable  to  software  related  to  system  diagnostics  or  system 
performance. 

(3)  Identify  and  record  software  revisions  during  testing  and  when  it  was  changed. 

(4)  Outlining  when  a  new  version  of  the  software  is  installed  and  why,  and  what  did 
the  revision  correct/enhance/provide. 

5.  DATA  REQUIRED. 

a.  The  data  requirements  are  identified  in  the  planning  phase  from  system  specifications 
and  critical  operational  requirements.  The  requirements  are  documented  in  the  test  plan  and 
concurred  by  stakeholders  before  the  test  starts.  Data  requirements  should  be  identified  for  each 
subtest  to  reflect  information  necessary  for  diagnostics  and  for  the  analysis  of  the  SUT  essential 
features.  Si-units  should  be  used. 

b.  The  following  data  elements  are  broken  down  into  the  various  subtest  data 
requirements.  The  test  team  will  record  the  following: 

5.1  Receiving/Initial  Inspections. 

a.  Inspections  prior  to  and  during  testing. 

b.  Operational  readiness,  safety  checks  and  inspections,  and  results  of  initial  functional 
check(s). 

c.  Description  of  packaging,  item  nomenclature  (serial  number  or  identifying  number), 
item  type  (if  special  variants  are  built),  and  quantity  of  each  type  of  test  item(s): 

(1)  Major  component  serial  numbers  and  proper  title  (name  of 
component)/nomenclature. 

(2)  Subcomponent  serial  numbers  and  proper  title  (name  of  component) 
/nomenclature. 

(3)  Cross  reference  to  any  local  numbers/codes  applied  for  testing. 

d.  Descriptions  and  photographs  of  shipping  configuration  packaging  and  detection 
system  damage  or  discrepancies. 


47 


TOP  04-2-090A 
8  January  2015 

e.  Document  completeness  of  the  SSP  received  for  the  test  (manuals,  repair  parts,  tools, 

etc): 

( 1 )  Calibration  tools/ devices . 

(2)  Batteries. 

(3)  Battery  Chargers. 

(4)  Cables. 

(5)  Calibration  pieces. 

f.  Record  component s)  operational  status,  which  items  needed  maintenance,  and 
software  upgrades.  Record  final  software  version  and  all  upgrades  implemented  during  testing. 

g.  Record  of  hardware/software  adjustments/modifications  made  prior  to  and  during 
testing. 

h.  Record  any  safety  or  human  factors  problems. 

i.  Document  compliance/noncompliance  with  test  criteria  or  hardware  specifications. 

j.  Identify  radar  operational  frequencies  (NOTE:  radar  frequencies  are  provided  to 
range  frequency  spectrum  managers  to  acquire  frequency  allocations  to  permit  authorization  to 
emit  for/during  testing). 

5.2  Physical  Characteristics. 

Record  physical  characteristics  prior  to  the  start  of  testing  with  a  representative  sample  of  SUT 
configuration  provided/received  for  testing: 

a.  Weights  of  all  major  components  and  shipping  containers  (±  0.001  kilogram  (kg)). 

b.  Dimensions  of  all  major  components  and  shipping  containers  (±0.1  cm). 

c.  Physical  characteristics  of  the  SUT  in  the  stowed  and  deployed  configurations. 

d.  Center  of  gravity  of  SUT  in  various  adjustable  configurations,  as  required. 

5.3  Performance. 

5.3.1  Common  Data  Requirements  for  All  Trials  (Mission  Runs). 

5. 3. 1.1  Administrative  Data. 
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a.  Trial  ID  number. 

b.  Trial  date. 

c.  Test  player  ID  number. 

d.  Test  player  equipment  and  clothing  (as  appropriate,  for  collation  to  human  factors 
appraisal). 

e.  System  ID  number. 

f.  Test  player  team  ID  number,  if  teams  of  individuals  are  used  for  search  method. 

5. 3. 1.2  Ground  Truth  Data. 

a.  Target  description  data. 

(1)  Record  of  each  unique  target  ID  number. 

(2)  Tabulated  composition  of  each  target  type. 

b.  Collect,  measure,  and  record  the  items  as  following: 

(1)  Configuration  of  every  threat  and  clutter  target. 

(2)  Surveyed  coordinate  position  of  every  target  (ground  truth). 

(a)  Exact  coordinate  location  of  every  target. 

(b)  Depth. 

(c)  Plotted  ground  truth  (spreadsheet  and  graphical  image). 

c.  Identify  target  code  categories  to  maintain  data  unclassified  (see  Appendix  C  for  target 
categories). 

5.3. 1.3  Mission  Data  for  Each  Trial  (Mission  ID). 

For  each  detection  mission,  log  the  test  lane  identification  number  and  the  administrative  data 
listed  in  paragraph  5. 3. 1.1,  as  well  as  the  following  data: 

a.  Run  Start  Time;  Detector  ON  time. 

b.  Run  Stop  Time;  Detector  OFF  time. 


49 


TOP  04-2-090A 
8  January  2015 

c.  Start  Point  (geodetically  marked)  (beginning  of  lane). 

d.  Geodetic  path  of  lane. 

e.  Stop  Point  (geodetically  marked)(end  of  lane). 

f.  Detection  alert  number  over  the  entire  lane. 

g.  Time  of  each  alert. 

h.  Alert  status  (e.g.,  targets,  clutter,  false  alann,  etc.). 

i.  Detection  alarm  sensor  (e.g.,  metal  detector,  GPR,  etc.)  when  each  sensor  is  activated, 
individually  (de-coupled)  or  combined. 

j.  Alarm  type  (e.g.,  audible,  LEDs,  etc.). 

k.  Power  Supply  (battery  use  information): 

(1)  Battery  type. 

(2)  Battery  ID. 

(3)  Battery  start  and  stop  time. 

(4)  Provide  an  overview  of  the  supplied  power  supply  to  assess  battery  power  supply 
availability  (e.g.,  battery  logistic  supply,  battery  power  charging  requirements,  NATO  adapters, 
etc.). 

(5)  Time  duration  SUT  will  operate  with  single  set  of  MIL-SPEC  batteries  before 
giving  a  “Low  Battery”  warning. 

(6)  Once  the  Low  Battery  warning  is  activated,  determine  duration  how  long  detector 
stays  operational  before  a  “Critical  Low”  warning  is  given  or  SUT  turns  off. 

(7)  Calculated  average  battery  life  when  using  the  SUT. 

l.  Detection  Alert  Confident  Level  (High  or  Low). 

m.  Results  of  performance  detection  test. 

(1)  Quantity  of  alert  markers  used  during  each  mission. 

(2)  Number  of  detections. 

(3)  Number  of  false  alarms. 
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(4)  Time  to  complete  each  test  area/test  route. 

(5)  Elapsed  time  SUT  was  on. 

n.  Failures  found  by  any  means,  including  during  PMCS,  BIT,  or  missions. 

o.  Results  of  all  BIT. 

p.  Meteorological  data  (regardless  of  whether  natural  conditions  or  artificially  developed 
conditions): 

(1)  Air  temperature. 

(2)  Precipitation,  duration,  intensity,  accumulation,  and  phase  (rain,  snow,  etc). 

(3)  Wind  speed/direction. 

(4)  Barometric  pressure. 

(5)  Relative  humidity. 

(6)  Ground  (soil)  temperature. 

(7)  Cloud  cover,  description/height. 

(8)  Time  of  Day. 

q.  Other  target  truth  data  unique  to  the  environment  or  detector  type  (temperatures  of 
backgrounds,  solar  loading  on  detector  components,  targets,  etc.). 

r.  Soil  data  (from  each  test  lane). 

(1)  Soil  characteristics  and  classification. 

(2)  Sampling  equipment  and  methods  used  to  characterize  soil  type. 

(3)  Metal  content. 

(4)  Soil  composition. 

(5)  Soil  texture. 

(6)  Rock  distribution. 

(7)  Moisture  content  and  method  used  to  characterize  moisture  or  control  moisture. 
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(8)  Vegetation  type  with  narrative  description,  photographs,  and  physical  survey 
locations  within  the  lane  boundaries. 

NOTE:  Identify  any  other  unique  soil  properties  or  soil  features  that  may  be  of  interest  for 
specific  sensor  types. 

5.3.2  Sensor  Perfonnance  and  Recommended  Analytical  Methods. 

Sensor  methods  are  continuously  evolving  in  the  attempts  to  standardize.  Certain  key  definitions 
of  interest  for  the  evaluation  of  sensor  performance  are  presented  in  paragraphs  5.3.2.a(l)  -  (7). 
Sensor  perfonnance  analytical  methodologies  are  presented  in  paragraphs  5. 3. 2. a,  b,  and  c,  to 
provide  a  starting  point  for  detection  analysis  of  detection  data  in  a  manner  relevant  to 
dismounted  detection  systems/operations. 

a.  For  detection  systems,  other  than  mine  detection  systems,  the  term  target  may  be  used 
interchangeably  with  mine  and  pressure-plate  improvised  explosive  device  (PPIED)  components 
(e.g.,  trigger  mechanisms,  command  wire,  and  main  charges,  etc.)  without  affecting  the 
definitions  or  analytical  methods  below. 

(1)  ROA  =  Rate  of  Advance  =  Area/Unit  of  Time  (Dismounted  Systems)  (nr/hr). 

(2)  ScR  =  Scan  Rate  =  Area  Scanned  or  Covered/Unit  of  Time  (nf/hr). 

(3)  SwR  =  Sweep  Rate  =  Distance  Traveled  (lateral  sweep)/Unit  of  Time  (m/hr); 

AR  =  Advance  Rate  =  Linear  Forward  Progress/Unit  of  Time  (nf/hr). 

(4)  Detection  Probabilities: 

Pd(tme)  =  Number  of  Targets  Detected/Total  Number  of  Targets  (N)seenote  1  (in  Lane) 
Pd(measwed)  =  Number  of  Declarations  that  meet  the  rcrit  (mine+haio)  criteria/ (N)see  note  2 
NOTE  1 :  Pd(tme)  implies  that  lucky  matches  may  occur. 

NOTE  2:  Pd(measured)  implies  that  due  to  target  merging,  two  targets  may  appear  as 
one  or  that  multiple  declarations  may  need  to  be  rescored  as  one  declaration  after 
review  of  truth  and  test  data. 

(5)  FAR  =  False  Alarm  Rate  =  False  Alarms/Unit  Area  or  False  Alarms/Linear 
Distance  along  a  specified  track  or  front  width. 

(6)  Pfa  =  Probability  of  False  Alarm. 

(7)  rcrit=Critical  Radius  or  "halo"see  note  3. 
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NOTE  3:  Each  detector  will  have  a  critical  radius  or  "halo"  in  which  a  detection 
can  occur  at  a  distance  relative  to  the  center  of  the  detector  head  (see  Figure  7). 
This  implies  that  a  detection  can  occur  with  a  standoff  distance  between  the  center 
of  the  detector  head  and  the  edge  of  the  target(s).  The  "halo"  performance 
envelope  is  determined  experimentally  for  each  integrated  sensor  type.  It  may  be 
circular  (implying  an  equidistance  in  all  directions)  or  perhaps  an  irregular 
footprint  area  when  overlaid  on  the  surface  of  the  ground.  Therefore,  rCnt  or 
"halo"  implies  the  distance  or  footprint  area  in  which  a  detection  alert  can  occur 
with  reasonable  probability.  Hence,  a  marked  detection  that  occurs  within  the  rcrit 
or  "halo"  is  scored  as  a  valid  alert  (see  Figure  7). 


Figure  7.  Target  halo  configuration. 


b.  Analytical  Scoring  Concept.  For  the  detection  alarms  obtained  during  the  scanning  of 
the  test  lane,  a  determination  must  be  made  regarding  the  “scoring”  of  reported  detections  as 
valid  or  invalid  (e.g.,  detection  or  false  alarm).  The  accuracy  criterion  will  be  used  to  make  this 
determination;  for  example,  if  the  location  of  the  detection  declaration  is  within  a  “halo”  about 
the  perimeter  of  the  target  that  is  the  distance  of  the  accuracy  requirement  away  from  the  target, 
the  detection  is  considered  valid  (see  Figure  7).  Thus,  all  valid  detections  will,  by  definition, 
meet  the  location  accuracy  requirement.  The  method  for  scoring  the  accuracy  of  results  will  or 
may  be  DGPS  survey  coordinate  system  overlaying  the  target  alarms  coordinates  to  the  ground 
truth  coordinates. 

c.  Definitions.  Although,  it  is  possible  for  there  to  be  more  than  one  accurate  detection 
of  a  single  target  (more  than  one  indicator  within  the  accuracy  requirement  (see  Figure  7, 
Redundant  Detection),  for  calculation  of  the  probability  of  detection  (Pd),  only  one  detection  per 
encounter  is  counted.  This  is  because  Pd  is  a  probability  which  must  be  accumulated  on  a  per 
encounter  basis;  if  the  total  number  of  accurate  detections  (which  could  be  more  than  one  per 
encounter)  and  the  total  number  of  encounters  are  merely  accumulated,  the  result  might  show  a 
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higher  Pd  than  was  actually  demonstrated  (the  calculated  Pd  could  conceivably  be  greater  than 
one).  However,  a  count  will  be  kept  of  the  number  of  additional  accurate  detections  (Aa)  per 
target  encounter  and  the  results  will  be  reported.  Although  this  is  or  may  not  be  directly  related 
to  a  user  requirement,  it  is  a  useful  measure,  in  addition  to  the  FAR,  of  the  degree  to  which  an 
overly  sensitive  system  might  impede  the  detector  and  operator’s  forward  movement.  It  is 
intended  that  Pd,  FAR,  and  Aa  be  analyzed  as  a  function  of  operator,  soils,  climate,  advance  rate, 
target  type(s),  etc. 

The  following  defined  parameters  apply  for  the  detection  tests: 

E  =  an  encounter  of  a  target  by  the  sensor  (IE=N) 

R  =  a  reported  detection 

A  =  an  accurate  detection;  i.  e. ,  a  reported  detection  within  the  required  accuracy 
(defined  by  the  critical  radii  (rCnt))  or  “ halo  ”  of  any  mine  detection  system) 

S  =  a  single,  “scored”  accurate  detection  for  a  single  encounter 
Aa  =  an  additional  accurate  detection 

FA  =  False  alarm:  a  reported  detection  outside  the  allowed  accuracy  (defined  by  the 
critical  radii  (rCnt))  or  “halo  ”  of  any  mine  detection  system) 

FAR  =  the  sum  of  FAs  per  linear  km  searched 

Thus, 

Pd*  =  IS/ IE  (where  Pd  <  1) 
lAa  =  I(A  -  S) 

IF  A  =  IR-IA 

FAR  =  IF  A/km  or  unit  area 

NOTE:  Pd  for  any  minefield  test  area  with  a  given  set  of  conditions  should  be  presented  in  test 
reports  as  a  numerical  ratio,  rather  than  a  point  estimate.  This  will  allow  for  statistical  tests  and 
combining  of  data  where  possible. 

5.4  System  Safety  and  Human  Health. 

a.  List  of  safety  hazards  from  the  TM/OM,  or  supplied  safety  assessment 
documents/analysis. 

b.  Results  of  SUT  safety  inspections  and  operational  checkout. 

c.  Warning  or  cautions  not  properly  labeled  on  detector  or  listed  in  the  SUT  TM/OM. 

d.  List  of  any  safety  hazards  observed  during  testing,  and  what  mitigating  steps  or 
protections  were  implemented  during  trials  or  proposed. 

5.5  Climatic  Suitability. 

Generic  data  requirements  are  as  follows  (if  applicable  to  a  specific  subtest): 
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a.  Ambient  air  temperature. 

b.  Conditioned  chamber  temperature. 

c.  Conditioned  water  temperature. 

d.  Time  versus  temperature  charts. 

e.  Solar  radiation  (irradiance  intensity). 

f.  Relative  humidity. 

g.  Salinity  of  solutions  (mineral  salts  used  for  testing). 

h.  PH  of  solutions. 

5.6  Transportation  and  Handling. 

a.  Testing  temperatures. 

b.  Time  versus  temperature  charts. 

c.  Pre-  and  post-photographs  of  testing  configuration  and  SUT. 

d.  Document  testing  configuration  (e.g.,  containerized,  bare,  deployed  or  stowed 
configuration,  etc.). 

e.  Document  testing  with  or  without  power  supply. 

f.  Vibration  spectrums  and/or  drop  heights. 

g.  Vibration  spectrum  versus  time  charts. 

h.  Identify  target  sample  for  operational  verification. 

i.  Document  any  test  or  operational  checkout  discrepancies. 

5.7  Integrated  Logistics  Supportability. 

a.  Photographs  of  the  shipping  and/or  packaging  configuration(s). 

b.  Physical  weights  and  dimensions  of  the  shipping/packaging  configuration(s). 

c.  Document  any  human  factors  handling  issues  in  the  use  and  handing  of  the  SUT 
interface  with  shipping/packaging  contained s). 
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d.  Document  the  TM/OM  publication  (or  draft)  date  and  version  number. 

e.  List  the  TM/OM  discrepancies;  identify  the  TM/OM  safety  discrepancies  and  what 
mitigating  measures  were  implemented  for  test. 

f.  Identify  any  special  training  procedures  that  were  implemented  for  assessment  of  the 
SUT,  and  any  procedures  that  were  not  included  that  would  be  vital  for  military  users. 

g.  List  the  spare  parts,  tools,  special  equipment,  etc. 

h.  Identify  the  type  and  quantities  of  the  battery(s)  used  for  test. 

i.  Identify  disposal  requirements  for  batteries. 

5.8  Reliability.  Availability,  and  Maintainability  (RAM). 

a.  RAM  data  elements  to  document  during  testing  that  should  be  considered: 

(1)  Record  of  scheduled  and  unscheduled  maintenance  activity. 

(2)  Time  for  typical  PMCS. 

(3)  Maintenance  task(s)  performed. 

(4)  Clock  hours  of  maintenance  task(s)  performed. 

(5)  Record  of  SUT  failures,  malfunctions,  and/or  errant  behavior. 

(6)  Operator’s  corrective  action  to  fix  a  detector  operational  issue,  and  time  for  fix 

(minutes). 

(a)  Identification  and  nomenclature  of  parts  involved  in  maintenance  action. 

(b)  Reason  for  maintenance  action,  or  action  taken  to  get  SUT  operational  (repair, 
replace,  adjust,  calibrate). 

(7)  Adequacy  of  manuals  (legibility,  readability,  applicability,  completeness) 
to  include  handling  instructions. 

(8)  Adequacy  of  instructional  training  for  scheduled  and  unscheduled 
maintenance  tasks,  if  applicable. 

(9)  Adequacy  to  access  the  components  for  operator  level  maintenance  tasks. 

b.  Other  maintainability  data  elements  for  consideration: 
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(1)  Adequacy  of  supplied  or  TM/OM  identified  special  tools,  test,  measurement,  or 
diagnostic  equipment. 

(2)  Document  how  long  the  battery  lasts  before  needing  replacement  during  missions 
(e.g.,  minutes,  and/or  hours). 

(3)  Assess  the  TM/OM  for  publication  version  updates  (e.g.,  does  it  include  latest 
versions  of  the  SUT  modifications,  enhancements,  software  control  elements,  etc.). 

5.9  Human  Factors  or  Ergonomics. 

The  following  data  are  required  for  human  factors  related  assessment: 

a.  Anthropometric  and  demographic  data  of  the  test  participants  (operators). 

b.  Document  the  skill  level  of  the  operators  (previous  use  of  detectors). 

c.  Photographs  of  operators  holding  equipment  in  packaging  and  bare  configurations,  to 
include  assembly  and  assembled  modes. 

d.  Record  the  level  and  duration  of  training  provided,  and  document  operator’s 
comments  on  the  training  and  their  confidence  level  after  the  training  in  questionnaires  and 
operator  interviews. 

e.  Record  the  safety  checklists  and  identify  the  hazards. 

f.  Document  the  operator’s  ability  to  perform  key  operational  functions  of  assembly, 
operate,  scan,  and  disassembly  of  the  SUT. 

g.  Document  fatigue  level  observed  while  operator  performed  mission  scanning 
operations. 

h.  Photograph  operator  with  ear  piece,  if  applicable. 

i.  Results  of  visual  and  auditory  assessments  of  the  SUT  operational  indicators  (e.g.,  ear 
piece,  speaker,  LED,  backlighting,  etc.). 

j .  Assessment  results  of  the  compatibility  and  adaptability  of  operator  when  using 
personal  protection  equipment  and  tactical  mission  apparel  (e.g.,  MOPP  and  NBC  gear,  gloves, 
back  packs,  ammo  vests,  head  protection,  goggles,  etc.). 

k.  Assessment  of  human/machine  interface  correlating  to  hardware  design  and 
ease/difficulty  in  operational  use. 
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l.  Record  the  duration  operators  are  scanning  with  the  SUT  between  breaks,  and  if 
applicable,  the  duration  of  rest  periods. 

m.  Record  and  summarize  the  questionnaire/interview  results. 

5.10  Electromagnetic  Environmental  Effects  (E3)  and  Vulnerability. 

a.  Results  of  E3  tests  performed  as  screening  tests  separate  from  field  trials  (E3  radiated 
emissions,  compatibility,  interference,  conducted  emissions,  radiated  susceptibility,  etc). 

b.  Identify  any  vulnerabilities  or  susceptibilities  of  equipment  to  RF  frequency  intrusion 
(deliberate  or  unplanned). 

c.  Identify  any  vulnerability  to  natural  phenomena  (lightning,  electrostatics). 

d.  Identify  susceptibilities  to  common  battlefield  RF. 

e.  Determine  minimum  separation  distance  between  two  detectors  or  detection-systems 
before  detectors  start  to  interfere  (electromagnetic  interference)  with  each  other. 

5.11  Software. 

a.  Record  of  software  version  at  start  of  system  performance  trials. 

b.  Record  of  problems  attributable  to  software  related  to  system  diagnostics  or  system 
performance. 

c.  Identify  and  record  software  revisions  during  testing  and  when  during  testing  it  was 
changed. 

d.  Document  at  software  version  at  the  end  of  testing. 

6.  PRESENTATION  OF  DATA. 

a.  All  raw  test  data  must  be  recorded  and  retained  for  a  specified  period  after  the  test. 
This  data  will  not  necessarily  be  presented  but  must  be  available  to  verify  the  test  results  if  there 
are  subsequent  queries. 

b.  The  results  of  the  test  should  be  recorded  in  a  Test  Report.  The  method  of  presenting 
and  the  level  of  detail  presented  will  be  directed  by  the  authority  for  whom  the  trial  is  being 
conducted  but,  as  a  minimum,  should  include: 

(1)  Descriptions  of  the  inspection,  specific  test  procedures,  and  results  for  each  SUT 
using  narration,  tables,  photographs,  charts,  and  graphs  as  appropriate. 
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(2)  Diagrams  and  photographs  (as  necessary)  to  show  the  type  of  container  (when 
applicable),  type  of  carrier  (attachable  equipment  only),  configurations  of  the  SUT  (e.g.,  stowing, 
assembled,  disassembled,  etc.),  and  to  document  any  damage. 

(3)  Performance  scoring  is  presented  in  tabular  tables  presented  by  various 
parameters.  The  analysis  of  the  data  collected  should  include  scores,  results,  and  any  other 
relevant  assessments.  Detector  perfonnance  should  be  analyzed  though  the  scoring  of 
performance  of  detection  and  fails  alann  rate,  and  ROC  curves  (see  Appendix  G).  Perfonnance 
must  also  be  broken  into  several  categories,  e.g.,  by  target  type,  depth,  operator,  mode  of 
detection,  lane  number,  etc.  The  scoring  must  include  a  study/comparison  of  the  difference  in 
performance  of  any  different  systems/platforms  as  determined  by  target  types  and  depths. 

(4)  Summary  of  results  should  include  a  description  of  the  perfonnance  of  missions 
with  respect  to  encounters,  perfonnance  of  detection,  alarm  rates,  advance  rate,  and  failures. 
Reliability  indices  such  as  operational  run  times,  maintenance  ratios,  and  availability  of 
equipment  should  be  computed. 
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APPENDIX  A.  FREQUENCY  ALLOCATION  FORM. 


FREQUENCY  ALLOCATION  INFORMATION  REQUIREMENTS 

(1) 

Name  of  Program 

(2) 

Purpose  of  Program 

(3) 

Security  Classification 

(4) 

Frequency 

(5) 

Transmitter  Power  (Watts) 

(6) 

Time  of  Usage 

(7) 

Required  Start  and  End  Dates 

(8) 

Transmitter  Nomenclature 

(9) 

Transmitter  Location 

(10) 

Transmitter  Antenna  Data 

(a)  Type/Name 

(b)  Gain 

(c)  Site  Elevation 

(d)  Antenna  Feedpoint  Height 

(e)  Orientation 

(f)  Polarization 

(11) 

Receiver  Nomenclature 

(12) 

Receiver  Location 

(13) 

Receiver  Antenna  Data 

(a)  Type/Name 

(b)  Gain 

(c)  Site  Elevation 

(d)  Antenna  Feedpoint  Height 

(e)  Orientation 

(f)  Polarization 

(14) 

J/F-12  Number  (if  assigned) 

Reference: 

Spectrum  Management  Office  (SMO)  Data  Requirements 

AR  5-12,  Army  Management  of  the  Electromagnetic  Spectrum  Oct  1997 

DD  Form  1494  Preparation  Guide 

Figure  A-l.  Typical  frequency  allocation  request  form. 
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APPENDIX  B.  DOCUMENT  REVIEW  CHECKLIST. 

B.l.  ATEC  TEST  CHECKLIST  (reference  DA  PAM  73-1). 

In  order  to  properly  assess  the  safety,  perfonnance,  and  reliability  of  the  SUT,  a  pre-planning 
detailed  review  must  be  conducted.  This  review  ensures  a  proper  overview  of  required 
administrative  and  safety  protocols  are  emplaced  prior  to  initiation  of  testing.  The  following 
documents  should  be  included  in  the  test  item  data  package;  it  is  preferable  that  these  documents 
be  provided  as  early  as  possible  prior  to  the  start  of  testing. 

a.  Safety  Assessment  Report. 

b.  Health  Hazard  Assessment  Report. 

c.  All  test  data  available  regarding  the  item  requiring  the  Safety  Release.  If  no  current  test 
data  are  available,  any  other  information  that  can  be  used  (for  example,  prior  Government  test 
data,  contractor  test  data),  with  the  emphasis  on  safety  data. 

d.  Environmental  documentation  (record  of  environmental  consideration  (REC)). 

e.  Training  plans. 

f.  Equipment  technical/operator  publications. 

g.  Mission  scenario/mission  profile. 

h.  Test  Plan. 

i.  Test  and  Evaluation  Master  Plan. 

j .  Frequency  Allocation  Documentation. 

k.  Software  Requirements  Specification. 

l.  U.S.  Army  Institute  of  Public  Health  Command  Study  (laser,  radiofrequency  radiation, 
and/or  ionizing  radiation). 

m.  System  Requirements  Documents. 

n.  Security  Classification  Guide,  and  Operations  Security  (OPSEC)  document. 

o.  Source  of  troops  involved  in  developmental  and  operational  testing. 

p.  Test  Readiness  Review. 
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APPENDIX  B.  DOCUMENT  REVIEW  CHECKLIST. 


NOTE:  When  sufficient  data  are  not  available  on  which  to  base  a  Safety  Release  or  complete 
safety  review,  additional  testing  may  be  necessary.  In  such  cases,  required  testing  will  be 
performed  by  the  testers,  and  test  costs  will  be  paid  by  the  materiel  developer  or  SUT  customer. 

B.2.  TEST  AND  EVALUATION  SUBTEST  OVERVIEW. 

Table  B-l  provides  a  summary  list  of  test  and  evaluation  subtests  that  should  be  considered  when 
designing  a  comprehensive  hand-held  detector  assessment.  Time,  cost,  and  SUT  resources 
determine  feasibility  of  execution  for  any  specific  test  acquisition  phase. 


TABLE  B-l.  TEST  AND  EVALUATION  SUBTEST  SUMMARY 


SUBTEST 

REFERENCE  TEST 
PROCEDURE  (S) 

DESCRIPTION 

Inspections 

Preliminary  Inspection 

Provides  inspection  and  baseline  operation 
prior  to  initiation  of  testing.  Ensures  test  item 
is  safe  and  ready  to  initiate  testing. 

Physical  Characteristics 

Determines  dimensions  and  other  physical 
characteristics  of  the  test. 

Final  Inspection 

Provides  inspection  and  baseline  operation 
following  completion  of  testing 

Performance 

Software  Performance 

ITOP  01-1-0569 

Used  to  evaluate  a  system’s  software 
functional  capabilities.  It  does  not  specifically 
address  other  software-related  issues,  such  as 
safety  or  security.  The  method  for  undertaking 
the  software  performance  assessment  discussed 
in  this  document  addresses  software  T&E  as  an 
integral  element  of  system  T&E  and  is  targeted 
at  the  system  performance  level.  Key  elements 
of  this  approach  include  the  allocation  of 
system  requirements  to  software,  assessment  of 
software  performance,  and  assessment  of  the 
impact  of  software  on  overall  system 
performance. 

Detection  Sensors 

-  metal 

-  GPR 

-  Infrared 

Validates  the  adequacy  and  accuracy  of 
detection  system  sensors. 
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TABLE  B-l.  CONTINUED 


SUBTEST 

REFERENCE  TEST 
PROCEDURE  (S) 

DESCRIPTION 

Logistics 

Logistics  Supportability 

Determines  the  logistic  supportability  of  the 
test  item  through  quantitative  and  qualitative 
analysis  of  the  test  item. 

Reliability,  Availability, 
Maintainability 

Quantitative  analysis  of  data  collected  during 
endurance  testing  in  order  to  determine  the 
overall  ability  of  the  system  to  complete  tasks. 

Technical  Manuals 

Determines  adequacy  and  accuracy  of  provided 
system  technical  manuals. 

BIT/Built-In  Test  Equipment 
(BITE)  Embedded 

Diagnostics 

Used  to  evaluate  system  particular  test, 
measurement,  and  diagnostic  equipment.  This 
test  also  takes  into  consideration  not  only  the 
interface  between  the  test  equipment  and  the 
system,  but  also  the  interface  between  the  test 
equipment  and  other  elements  of  the  planned 
maintenance  support  such  as  manuals,  repair 
parts,  common  test  equipment  and  tools,  and 
calibration  facilities,  etc. 

Safety/Software 

System  Safety 

TOP  01-1-06010 
TOP  02-2-50811 

Used  to  identify  and  evaluate  hazards 
associated  with  test  items.  Testing  will  provide 
determination  or  assessment  of  personnel  and 
equipment  hazards  in  the  system  and 
associated  operation  and  maintenance  hazards. 

Critical  Software  Analysis 
and  Testing 

ITOP  01-1-05712 

Describes  the  activities  necessary  to  ensure  that 
safety  is  designed  into  software  that  is  acquired 
or  developed  and  that  safety  is  maintained 
throughout  the  software  life  cycle.  It  provides 
uniform  procedures  for  developing  and 
implementing  a  safety-critical  software  test 
methodology  of  sufficient  comprehensiveness 
to  identify  the  software  caused  hazards  of  a 
system  and  to  impose  design  requirements  and 
management  controls  to  prevent  mishaps.  The 
objective  is  to  ensure  that  the  software  design 
takes  positive  measures  to  enhance  system 
safety,  and  that  software  errors  which  could 
reduce  system  safety  have  been  eliminated  or 
controlled  to  an  acceptable  level  of  risk. 
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TABLE  B-l.  CONTINUED 


SUBTEST 

REFERENCE  TEST 
PROCEDURE  (S) 

DESCRIPTION 

Human  Machine  Interface 

Human  Factors 

TOP  01  -2-6 1013 

Used  to  provide  human  factors  engineering 
assessment  of  equipment. 

Noise  Levels 

MIL-STD-1474D14 
DA  PAM  40-501 15 

Used  to  determine  operating  noise  levels. 

Environmental 

Environmental  Performance 

-  High  Temperature 

-  Low  Temperature 

-  Solar  Radiation 

-  Rain 

-  Humidity 

-  Fungus 

-  Salt  Fog 

-  Sand  and  Dust 

-  Icing/Freezing  Rain 

-  Vibration 

MIL-STD-810G 

Determines  the  operating,  maintenance,  and 
durability  characteristics  the  detection  system 
when  operating  in  extreme  environments. 

E3 

EMI/EMC 

-  Radiated  Emissions 

-  Radiated  Susceptibility 

-  Hazards  of  Electromagnetic 
Radiation  to  Ordnance 
(HERO) 

-  Hazards  of  Electromagnetic 
Radiation  to  Fuel  (HERF) 

-  Hazards  of  Electromagnetic 
Radiation  to  Personnel 
(HERP) 

-  Helicopter  Electrostatic 
Discharge  (HESD) 

-  Personnel  Electrostatic 
Discharge  (PESD) 

-  High  Altitude 

Electromagnetic  Pulse 
(HEMP) 

-  Near  Strike  Lightning 

-  Bonds  and  Grounds 

-  Conducted  Susceptibility 

-  Conducted  Emissions 

-  Intra-system  EMC 

TOP  01-2-511  A16 

Determines  whether  the  item  tested  meets  the 
electromagnetic  radiation  effects,  static 
electricity,  and  lightning  criteria  and  the 
maximum  electromagnetic  radiation 
environment  to  which  the  test  item  may  be 
exposed  without  adverse  effects.  Ensures  that 
the  equipment  under  test  is  able  to  operate  in 
its  intended  electromagnetic  environment 
without  its  performance  being  degraded  and 
without  degrading  the  performance  of  other 
system(s)  in  close  proximity 
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TABLE  B-l.  CONTINUED 


SUBTEST 

REFERENCE  TEST 
PROCEDURE  (S) 

DESCRIPTION 

Electromagnetic  Interference/Compatibility 

Interference 

MIL-STD-461F17 

Provides  verification  requirements  for  the 
control  of  the  electromagnetic  interference 
(emission  and  susceptibility)  characteristics  of 
electronic,  electrical,  and  electromechanical 
equipment  and  subsystems. 
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C.l.  SUPPLEMENTAL  PLANNING. 

This  appendix  contains  outline  examples  that  can  aid  in  the  organization  of  those  processes 
associated  with  planning  and  execution  of  trials  associated  with  detection  testing.  Development 
of  the  scope  of  the  trials  as  well  as  logical  organization  of  data  collection  is  critical  to  an  orderly 
examination  of  any  technology.  Table  C-l  lists  examples  of  threat  target  types  that  may  be  used 
for  detection  assessment  trials.  Figure  C-l  shows  a  typical  target  characterization  data  form. 


TABLE  C-l.  DETECTION  THREAT  TARGET  TYPES 


TARGET 

CATEGORY  /  DESCRPTION 

Mines 

Anti-tank  metallic  (AT-M) 

Anti-tank  low  metallic  (AT-LM) 

Antipersonnel  low  metallic  (AP-LM) 

Antipersonnel  metallic  (AP-M) 

IED  Arrays  (complete 
system) 

None 

IED  Component  -  Pressure 
Plates 

-  Low  Metal  Pressure  Plates  (LMPP); 

-  No  Metal  Pressure  Plates  (NMPP); 

-  High  Metal  Pressure  Plate; 

-  Fuzing  Devices:  Low  metallic  pressure  plates  (carbon  core, 
tire),  non-metallic  pressure  plates  (fence  post,  double 
plunger),  lamp  cord,  and  enamel  coated  copper  wire. 

IED  Component  -  Main 
Charges 

-  No  Metal  Main  Charge  (NMMC):  Ammonium  nitrate-filled 
plastic  jug  (non-metallic),  ammonium  nitrate-filled  plastic  pail 
(non-metallic);  mine. 

-  Metal  Main  Charge  (MMC):  mine  and  ammonium  nitrate- 
filled  pressure  cooker  (metallic). 

IED  Component  -  Other 

command  wire 

Clutter 

manmade  and  natural  (metallic  and  non-metallic) 

NOTE:  Included  in  any  target  set,  the  emplacement  of  clutter  must  be  considered  to  simulate 
urban  environmental  ground  conditions  the  detection  systems  may  be  exposed  to  during  a 
clearance  mission.  Clutter  types  for  this  test  are  categorized  as  metallic  or  nonmetallic.  All 
targets  are  buried  at  standardized  depths  for  that  specific  target  category. 
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APPENDIX  C.  SUPPLEMENTAL  PLANNING  AND  RANGE  CHARACTERIZING. 

C.2.  TEST  LANE  CHARACTERISTICS. 

a.  Table  C-2  provides  examples  of  soil  and  topography  characteristics. 


TABLE  C-2.  EXAMPLES  OF  SOIL/TOPOGRAPHY  CHARACTERISTICS  FOR  TRIALS 


SOIL 

PROPERTIES 

SOIL  TYPE  -  UNIFIED  SOIL  CLASSIFICATION  SYSTEM  (USCS) 

Clayey  Sand 
(USCS  Type  SC) 

Clay  of  Low  Plasticity 
(USCS  Type  CL) 

SILT 

(USCS  Type  ML) 

Surface  roughness 

Smooth  (<  5  cm) 

Medium  (<  10  cm) 

High  (>  1 0  cm) 

Contamination 

Metal  content 

Ferrous 

Non-ferrous 

Water  content 

<5% 

<  10% 

>  10% 

Rocks 

<  7  cm 

>  7  cm 

Debris 

<  7  cm 

>  7  cm 

NOTE:  Develop  a  table  for  soil  classifications  required  for  the  application.  Prioritize  those  soil 
types  from  highest  to  lowest  probability  of  encounter.  Select  trials  sites  based  upon  that 
examination  to  provide  a  broad,  but  representative  sampling  of  soil  conditions  relevant  to  the 
examination.  The  soil  considered  for  the  tests  could  be  different  types,  according  to  the  principal 
component  (sand,  clay  or  silt).  In  this  way  we  satisfactorily  cover  a  wide  range  of  realistic 
significant  cases  for  soil  conditions.  For  a  better  understanding  of  soil  properties,  the  test  team 
should  investigate  expertise  existing  on  site,  leverage  existing  soil  topographical  and  soil  surveys 
of  sites,  and/or  confirm  soil  types  and  topography  through  a  pre-trials  survey.  Where  possible, 
universal  or  uniform  soil  classifications  should  be  used  in  conjunction  with  a  narrative 
description  of  the  soils  to  pennit  comparison  with  other  trials. 
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b.  This  is  an  example  of  a  descriptive  write  up  without  detail  soil  characteristics  data: 

The  Detection  System  X  will  perform  the  detection  phase  on  test  lanes  at  the  Countermine 
Testing  and  Training  Range  (CTTR),  in  the  Sonoran  Desert  of  Arizona.  The  Detection  System  X 
off  route  detection  blind  test  lanes  consisted  of  1.5-meter  wide  and  50-meter  long  test  lanes. 

They  are  constructed  on  dirt  trial  over  flat  terrain  and  traversing  in  and  out  of  wash  beds  (wadi). 
The  lanes  are  categorized  as  calibration,  data  collection,  blind,  and  excursion. 
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APPENDIX  D.  SUPPLEMENTAL  DATA  REQUIREMENTS  AND  FIELD  DATA  SHEETS. 

D.l.  FIELD  DATA  SHEETS. 

Figures  D-l  and  D-2  are  examples  of  typical  field  data  collection  sheets. 


Detector  Data  Sheet 

RUN  Number _  DATE: _ 

OPERATOR _  Lane:  _  DATA  COLLECTOR: _ 

Detector  SERIAL  NUMBER: _ 

ENVIRONMENTAL  CONDITIONS:  □  Sunny  □  Hot  □  Nice  □  Cold  □  Windy  □  Dusty  □  Overcast  □  Rain  □  Night 

Other: _ 

START  TIME: _  FINISH  TIME: _  BREAK  TIME: _ ELAPSED  TIME: _ 


Poker  Chip 
Targets  =  Blue/Red 
Clutter  =  White 

Alarm/Sensor  Modal 

(check  as  many  that  ap 

ty 

ply) 

Comments 

Number 

Color 

MD 

PENTAD 

IED 

WIRE 

SAFETY  AND  HUMAN  FACTORS  OBSERVATIONS  ( X  on  seloctod  number;  comments  as  nocdod  and  on  back) 

o  Scanning  Comfort  Scale:  Extreme  Discomfort  ®  ®  (§)  (4)  (5)  (6)  (7)  (§)  (§)  ®  Comfortable 

If  Less  than  7  explain,  why: _ 

o  Operational  Use: 

□  Earpiece  □  Loud  Speaker  Sound  Level  Low  ®  (2)  (3)  (4)  (5)  (6)  (7)  (8)  (9)  ®  High 

□  LED  Visual  Illumination  Level  Poor  ®  (2)  (§)  (4)  (5)  ®  (7)  (8)  (§)  ®  Great 

□  Disassembly  □  Assembly  Difficult  ®@®®®®@®®®  Easy 

If  Less  than  7  explain: _ 

o  Data  Collector's  Observations  of  Operator  Or  Safety  Issues: _ 


Page _ of 


Figure  D-l.  Example  of  a  field  data  collection  sheet. 
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BATTERY  MONITORING  DATA  SHEET 


Detector  (Unit: ) 


Battery  SN 

Date  On 

Time  On 

SOC  ON 
(1-4/  2-5) 

Time  Off 

SOC  OFF  (1 

4/2-5) 

Legend:  SOC  —  State  of  Charge  (if  dispayed) 


Figure  D-2.  Example  of  a  field  data  collection  sheet. 
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APPENDIX  D.  SUPPLEMENTAL  DATA  REQUIREMENTS  AND  FIELD  DATA  SHEETS. 

D.2.  OVERVIEW  OF  FIELD  DATA  CONSIDERATIONS. 

Table  D-l  shows  typical  Mission  or  Trial  Infonnation. 


TABLE  D-l.  SYSTEM  PERFORMANCE  TRIALS  DATA  REQUIREMENTS 


DATA  ELEMENTS 

FOOTNOTES 

1 .  Administrative  Data 

1 . 1  Trial  (or  Mission)  ID  number 

1.2  Trial  (or  Mission)  Date 

1.3  System  ID  number 

1 .4  Platform  ID  number  (if  vehicle  platform) 

1 .5  Test  Player  ID  number 

1.6  Test  Player  Team  ID  number  (if  teams  of  individuals  are  used);  list  ID  numbers 
of  all  test  players  on  team 

1.7  Test  Player  Equipment,  Clothing,  and  Accessories  (as  appropriate,  for  collation 
to  human  factors  appraisal) 

•  Clothing  (Army  Combat  Uniform  (ACU),  Cold  Weather  clothing,  or 

Mission  Oriented  Protective  Posture  (MOPP)  level) 

•  Accessories  (Night  Vision  Goggles,  Protective  Glasses  or  Masks,  Radios, 
etc) 

1.8  Data  Collector  ID  number 

2  Mine/Test  Target  Data 

2.1  Target  ID  Number  (ITOP  04-2-521,  paragraph  5. a) 

2.2  Target  Location  (ITOP  04-2-521,  paragraph  5.b) 

2.3  Target  Burial  Depth  (ITOP  04-2-521,  paragraphs  5.c  and  5.d) 

2.4  Target  Orientation  (ITOP  04-2-521,  paragraph  5.e) 

2.5  Target  Weathering  Data  (ITOP  04-2-521,  paragraph  5.f) 

2.6  Target  Emplacement  Technique  (ITOP  04-2-521,  paragraph  5.g) 

2.7  Target  Characteristics  and  Features  (ITOP  04-2-521,  paragraph  5.i) 

2.8  Other  Target  Information  (ITOP  0  4-2-521,  paragraph  5.j) 

3  Target  Lane  Array  Data 

3.1  Mine  Lane  ID  number;  List  target  ID  numbers  of  all  mines  within  lane 
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TABLE  D-l.  CONTINUED 


DATA  ELEMENTS 

FOOTNOTES 

3.2  Geodetic  plots  and  x-y  position  spreadsheets  of  all  mines/targets  within  lane 

3.3  Overlay  of  Mine  Lane  ID  numbers  on  respective  lane  ID  numbers 

3.4  Updated  plots  and  position  spreadsheets  if  deliberate  changes  are  made  to  a  Lane 
ID,  such  as  planned  disturbance  or  addition  of  detritus  (battlefield  trash  or  other 
objects) 

3.5  Description  and  location  of  any  combat  obstacles  used  in  conjunction  with 
Lane/Target  Arrays  (ITOP  04-2-521,  paragraph  5.h) 

4  Meteorological  Data  (regardless  of  whether  natural  conditions  or  artificially 
developed  conditions)  on  inter-range  instrumentation  group  (IRIG)  basis  collated 
with  Trial  or  Mission  ID  number)  s) 

4.1  Air  Temperature 

4.2  Precipitation 

Duration 

Intensity 

Accumulation 

Phase  (rain,  snow,  etc). 

4.2  Wind 

Speed 

Direction 

4.3  Barometric  Pressure 

4.4  Relative  humidity 

4.5  Ground  Temperature 

4.6  Solar  Loading 

4.7  Cloud  cover,  height 

4.8  Light  level 

5  Soil  Data  (for  all  lane  ID  numbers) 

5.1  Geodetic  plot  of  soil  survey  measurement  points  and  tabulation  of  classification 
and  characterization  by  measurement  points. 
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TABLE  D-l.  CONTINUED 

DATA  ELEMENTS  FOOTNOTES 

5.2  Type  of  soil  and  soil  classification/sampling  system  or  method  used  to 
characterize: 

a.  Metal  content 

b.  Soil  composition 

c.  Soil  texture 

d.  Rock  distribution 

e.  Surface  roughness 

f.  Electrical  properties  (conductivity,  dielectric  constant,  etc.) 

g.  Magnetic  properties  (frequency  dependence) 

h.  Thermal  properties  (conductivity,  diffusivity,  temperature,  emissivity, 
density,  specific  heat) 

i.  Clutter  (natural  and  man-made;  magnetic  and  non-magnetic;  roots,  holes, 
wood,  plastic,  unusual  soil/overburden  stratification's  and  discontinuities).  If  clutter 
is  placed  as  a  target  document  1AW  ITOP  04-2-521  and  Mine/Test  Target  Data 

requirements. _ 

5.3  Moisture  content  and  method  used  to  characterize  moisture.  Description  of  any 

method  used  to  control  moisture. _ 

5.4  Other  ground  truth  data  unique  to  the  environment,  detector  type  or  detector 
platform  (such  as  vehicle  speed/position,  radiometric  temperatures  of  backgrounds 

and/or  calibration  targets). _ 


6  Vegetation/Terrain  Data 

6.1  Narrative  description,  photographs,  and  physical  surveys  of  terrain  and 
vegetation _ 


7  Mission  Data  (for  each  Trial  or  Mission  ID  number) 

7.1  Start  Time 

7.2  Stop  Time 

7.3  Start  Point  (Geodetically  Marked) 

7.4  Geodetic  Path  and  Swath  Path  in  Lane 

7.5  Stop  Point  (Geodetically  Marked) 

7.6  Alert  Number 

7.7  Time  of  Alert 

7.8  Alert  Status  (Initial  Alert,  Continue,  Disappear) 

7.9  Detection  Point  for  Alert  (Geodetically  Marked) 
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APPENDIX  D.  SUPPLEMENTAL  DATA  REQUIREMENTS  AND  FIELD  DATA  SHEETS. 

TABLE  D-l.  CONTINUED 


DATA  ELEMENTS 

FOOTNOTES 

8  Sensor  Experimentation  Testing  Data 

8.1  See  2. 1  through  2.7  for  common  data  requirements.  Although  not  all  may 
applicable,  these  will  provide  a  framework  for  collection. 

8.2  Description  of  test  setup  to  include  construction  materials  of  soil  bins,  special 
apparatus  (if  used)  to  control  detector  position  (x,  y,  z),  detector  advance  or  sweep 
rates,  and  detector  angular  displacement. 

8.3  Performance  and  Performance  Analysis.  Provide  IAW  Appendix  C. 

9  Sensor  Verification  Testing  Data 

9. 1  See  2. 1  through.7  for  common  data  requirements.  Although  not  all  may  be 
applicable,  these  will  provide  a  framework  for  collection. 

9.2  Performance  and  Performance  Analysis.  Provide  IAW  Appendix  C. 

10  Lane  Testing  Data 

10.1  See  5.2.1  through  5.2.7  for  common  data  requirements. 

10.2  Performance  and  Performance  Analysis.  Provide  IAW  Appendix  C. 

10.3  Measure  or  analysis  of  the  likelihood  of  a  mine  fuse  being  initiated  by  the 
detection  equipment 
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Trials  Information 


BB 

BB 


INSTRUCTIONS 

This  form  is  the  operator’s  opportunity  to  provide  feedback  about  the  unit  and  system.  Any  comments 
made  will  be  helpful  in  developing  a  system  that  will  perform  its  mission  safely  and  effectively,  and 
without  placing  undue  stress  or  hardship  on  the  system  operator. 

The  operator  is  asked  to  evaluate  each  item  on  this  form  and  give  a  rating  from  1  to  5,  with  the  higher 
numbers  being  an  easier/more  comfortable/better  score.  Room  is  available  on  the  last  page  for 
explanatory  comments.  The  operator  is  requested  to  explain  every  1  or  2  rating;  any  other  comments  are 


also  helpful. 

1.  System  Operation 

a.  Assembly: 

1 

Difficult 

2 

3 

4 

5 

Not  difficult 

b. 

Operation: 

1 

2 

3 

4 

5 

Difficult 

Not  difficult 

c. 

Alerts: 

1 

2 

3 

4 

5 

Could  not  understand 

Easily  understood 

Figure  E-l. 

.  Human  Factors  sample  survey. 

Test  Player  ID 
System  ID 


Date 
Trial  ID 
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d. 

Understanding  display: 

1  2 

Difficult 

3 

4 

5 

Not  difficult 

e. 

Changing  batteries: 

1 

2 

3 

4 

5 

Difficult 

Not  difficult 

2. 

Manpower  and  Personnel  Integration  (MANPRINT)/Human  Factors/Ergonomics 

(1)  Helmet: 

1 

2 

3 

4 

5 

Uncomfortable 

Comfortable 

(2)  Backpack: 

1 

2 

3 

4 

5 

Uncomfortable 

Comfortable 

(3)  Holding  wand: 

1 

2 

3 

4 

5 

Uncomfortable 

Comfortable 

(4)  Moving  wand  repeatedly: 

1 

2 

3 

4 

5 

Uncomfortable 

Comfortable 

Figure  E-l.  Human  Factors  sample  survey. 
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(5)  Audio  tones: 

1 

Harsh 

2 

3 

4 

5 

Comfortable 

(6)  Display: 

1 

Hard  to  Read 

2 

3 

4 

5 

Easy  to  read 

(7)  Controls: 

1 

Difficult 

2 

3 

4 

5 

Not  Difficult 

3.  Other 

(1)  Manuals  and  documentation: 

1  2 

Hard  to  follow 

3 

4 

5 

Complete,  useful 

(2)  Safety  during  operations: 

1  2 

Operator  is  not  safe 

3 

4 

5 

Operator  is  safe 

(3)  Built  In  Test  (BIT): 

1 

Difficult  or  incomplete 

2 

3 

4 

5 

Easy  and  complete 

Figure  E-l.  Human  Factors  sample  survey. 
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(4)  Daily  PMCS: 

1  2  3  4  5 

Difficult  or  incomplete  Easy  and  complete 


4.  Comments 

Item:  _ 

I  Comment:  T"" 


Figure  E-l.  Human  Factors  sample  survey. 
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Human  Factors  Questionnaire: 

1)  Ease  of  System  Operation. 

2)  Ease  of  use  and  functionality  of  audio/visual  displays 

3)  Clarity  and  distinction  of  audio/visual  alerts 

4)  Where  the  controls  performance  indicators  legible. 

5)  Ease  of  operator  troubleshooting  and  repairs. 

6)  Ease  of  the  system  transport  in  carrying  case,  assembly  of  system,  and  transport  of 
assembled  system. 

7)  Fit  and  comfort  of  components.  Include  comments  on  ease  of  performing  nonnal  tasks, 
ability  to  adjust  fit  of  components  to  operators  of  different  body  dimensions,  and  ease  of 
time  required  for  removal  of  components. 

8)  Any  difficulties  encountered  in  performing  any  mission  task  due  to  mission-required 
clothing  or  equipment. 

9)  Was  the  information  in  the  Operations  Manual  adequate  in  providing  instructional 
guidance  on  the  operation  and  maintenance  of  the  system? 


Figure  E-l.  Human  Factors  sample  survey. 
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APPENDIX  F.  EXCURSION  TRIALS  CONSIDERATIONS. 


This  Appendix  provides  sample  guidance  on  procedures  that  may  be  used  in  the  detectability 
performance  testing  of  detection  equipment  for  countermine  and  demining  equipment  (handheld 
dismounted  detection  sensors)  for  Excursion  trials. 

F.l.  LIMITATIONS. 

a.  This  Appendix  provides  a  narrative  outline  for  testing  of  portable  or  handheld  detection 
equipment  using  magnetic  induction  and  GPR  sensors.  With  review  and/or  slight  modification 
this  guidance  may  also  be  suitable  for  vehicle  mounted  detection  systems  and  detection  systems 
using  IR  or  other  sensors. 

b.  The  information  provided  is  for  testing  detection  equipment  in  Sensor  Experimentation 
Facilities.  With  review  and/or  slight  modification  this  guidance  may  also  be  suitable  for  testing 
in  sensor  verification  areas  or  target  lanes. 

F.2.  TESTING  OBJECTIVES. 

a.  Testing  objectives  will  impact  on  the  test  design.  Some  examples  of  testing  objectives 
are  as  follows: 

(1)  To  benchmark  system  or  subsystem  performance. 

(2)  To  determine  if  a  change  to  equipment  design  has  improved  perfonnance. 

(3)  To  determine  what  piece  of  equipment  performs  better. 

(4)  To  determine  if  a  piece  of  equipment  meets  minimum  performance  characteristics. 

(5)  To  determine  with  a  very  high  degree  of  confidence  the  exact  performance 
capabilities  of  the  equipment. 

(6)  Quality  control/sampling  type  testing. 

b.  Test  objectives  will  impact  on  what  test  data  should  be  collected  and  number  of  data 
points  required  to  achieve  the  desired  degree  of  statistical  confidence. 
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APPENDIX  F.  EXCURSION  TRIALS  CONSIDERATIONS. 


F.3.  EXCURSION  TESTING. 

F.3.1  Excursion  Sensing  Envelope  Testing. 

Record  equipment  response  when  passing  directly  over  a  surface  emplaced  target  at  the 
manufacturers  recommended  head  operating  height.  Repeat  test  passing  at  increasing  offset 
distances  from  the  target  centerline  until  the  target  is  no  longer  detectable.  Increase  distances 
from  the  target  centerline  in  either  2  or  4  cm  increments.  For  non- symmetrical  targets  repeat 
testing  along  other  target  axis’s.  Repeat  testing  with  different  size  targets  and/or  targets 
containing  different  amounts  of  metal. 

F.3.2  Excursion  Target  Depth  Testing. 

Record  equipment  response  when  passing  directly  over  a  surface  emplaced  target  at  the 
manufacturers  recommended  head  operating  height.  Repeat  test  with  the  same  target  at 
increasing  burial  depths  recommended  in  ITOP  04-2-521  until  the  target  is  no  longer  detectable. 
Repeat  testing  with  different  size  targets  and/or  targets  containing  different  amounts  of  metal. 

F.3.3  Excursion  Target  Size  Testing. 

Record  equipment  response  when  passing  directly  over  a  surface  emplaced  target  at  the 
manufacturers  recommended  head  operating  height.  Repeat  testing  with  different  size  targets 
and/or  targets  containing  different  amounts  of  metal  until  the  target  is  no  longer  detectable. 
Repeat  testing  at  increasing  burial  depths  recommended  in  ITOP  04-2-521. 

F.3.4  Test  Excursions. 

Conduct  the  above  tests  with  changes  to  the  following  variables: 

a.  Different  head/antenna  operating  heights  and  tilts. 

b.  Faster  or  slower  sweep  velocities. 

c.  Wet  detector  heads/antennas  (mist  or  immersion). 

d.  Batteries  in  varying  state  of  charge. 

e.  Time  or  temperature  effect  on  sensitivity  drift/stability. 

f.  Different  sensitivity  or  control  settings  on  the  equipment  under  test. 
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F.3.5  Soil/Overburden. 

F.3.5.1  Soil  Type  Selection. 

Depending  on  the  type  of  equipment  under  test,  repeat  above  testing  with  different  type  soils.  At 
a  minimum  consider  the  following  or  similar  type  soils: 

a.  Dry  sand. 

b.  Wet  clay/loam. 

c.  Magnetite  sand  (30%). 

F.3.5. 2  Soil  Conditions. 

Consider  testing  in  soils  ranging  from  very  dry  to  those  completely  saturated  from  high  water 
table  conditions.  Consider  that  some  detection  equipment  may  be  required  to  operate  in  soil  with 
high  levels  of  salts.  Soil  moisture  may  be  non-homogeneous.  Natural  or  artificial  ice/snow  can 
be  placed  on  the  soil.  The  soil  may  be  frozen  ranging  from  just  a  few  centimeters  to  far  below 
the  test  targets.  The  soil  surface  can  be  misted  or  soaked  to  simulate  conditions  shortly  after  a 
rainfall.  Consider  that  the  degree  of  soil  surface  roughness  can  strongly  influence  the 
performance  of  some  GPR  sensors.  Consider  that  the  detection  system  may  have  to  operate  on 
unimproved  roads  that  may  be  gravel  rather  than  soil.  Consider  that  the  soil  may  support 
vegetation. 

F.3.5. 3  Soil/Overburden  Clutter. 

Clutter  is  an  object  or  feature  that  interacts  with  a  detection  sensor  in  a  way  similar  to  or  identical 
to  a  mine.  Clutter  can  be  that  naturally  existing  at  the  test  area  or  emplaced  there  just  for  test 
purposes.  Clutter  can  be  natural  or  manmade.  Clutter  location,  like  target  location,  can  be 
documented.  However  if  the  test  site  selected  is  left  undisturbed  this  creates  a  dilemma  in  that 
the  natural  clutter  location  might  have  to  be  detennined  using  various  detection  sensors.  Clutter 
includes  pieces  of  magnetic  and  non-magnetic  metal,  roots,  holes,  rocks  of  various  composition 
and  size,  unusual  soil/overburden  stratifications  and  discontinuities,  wood,  plastic,  etc.  The 
discrimination  of  mines  from  clutter  is  perhaps  the  most  challenging  component  of  effective 
detection  system  design. 
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F.3.5.4  Soil  Condition/Clutter  Selection. 

The  ratio  of  probability  of  detection  to  false  alarms  rates  (Pd/FAR)  is  frequently  used  as  the  final 
measure  of  detection  system  performance.  This  ratio  is  primarily  dependent  on  the  detection 
equipment  capability,  its  sensitivity  settings,  the  type,  number  and  emplacement  of  targets  in  the 
test  area,  the  soil  or  overburden,  the  considerations  in  paragraph  2.4,  and  the  clutter.  To 
systematically  address  these  variables  it  is  recommended  that  detection  equipment  performance 
first  be  tested  and  baselined  in  homogenous  soil  conditions  (without  clutter  or  any  property 
stratification).  Keep  the  surface  soil  smooth  and  without  vegetation  or  clutter.  Address 
equipment  performance  and  test  excursions  (paragraph  2.4)  in  different  soil  types  and  moisture 
conditions  before  introducing  other  conditions  or  variables. 
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G.l  SCORING  FOR  DETECTION  AND  FALSE  ALARMS. 

a.  Using  the  alarm  files  of  potential  target  locations  are  either  matched  to  an  emplaced 
target  and  called  “detection,”  or  not  matched  to  a  target  and  considered  a  “false  alarm.” 
Declarations  are  matched  to  an  emplaced  target  if  the  declaration  is  within  a  critical  distance, 
Rhalo,  of  the  edge  of  the  target.  The  value  of  Rhalo  is  determined  through  the  analysis  of  the 
distances  between  all  of  the  alarm  declarations  and  the  true  target  locations  from  the  ground 
truth,  not  a  value  designated  prior  to  testing.  This  is  called  a  “miss  distance”  analysis. 

b.  This  distance  criterion  can  result  in  more  than  one  candidate  declaration  that  matches  a 
particular  emplaced  target.  When  there  are  redundant  declarations  within  the  Rhalo,  the  operator 
is  credited  with  a  single  detection  of  the  target.  Redundant  detections  are  not  counted  as  false 
alarms.  If  a  declaration  is  not  within  Rhalo  of  any  emplaced  target,  and  is  located  within  the  test 
lane,  then  that  declaration  is  considered  a  false  alann. 

c.  The  Pd  and  FAR  are  the  two  primary  measures  derived  from  a  blind  test.  The  detection 
probability  is  simply  the  fraction  of  the  encountered  targets  that  are  detected  by  the  operator(s). 

d.  This  probability  is  computed  for  each  target  category  as  well  as  for  each  target  type. 
This  number  reflects  a  “point  estimate”  of  the  probability  of  detection.  90%  confidence  intervals 
based  on  the  binomial  distribution  will  be  calculated  for  all  Pd’s. 

e.  The  FAR  is  the  measure  of  the  number  of  false  alarms  per  square  meter  of  operation  for 
the  detector.  Opportunities  for  false  alarms  occur  in  the  area  of  the  test  lane  that  does  not  include 
the  area  covered  by  the  targets  and  the  region  defined  by  their  Rhalo,  that  is:  test  lane  area  -  area 
of  targets  within  Rhalo. 

f.  Figure  G-l  shows  a  conceptual  scoring  example  showing  alarm  markers,  target  halos, 
and  targets,  to  include  detections,  redundant  detections,  and  false  alarms. 
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False  Alarm: 
Alarm  is  not  within 
0.76  meters 


Detect:  Alarm  is 
within  0.76  meters, 
and  line 

perpendicular  to 
direction  oftravel 
crosses  halo 


S3 


Single  False 
Alarm:  Two  or 
more  false  alarms 
within  0.76  meters 
of  each  other  are 
scored  as  one. 


False  Alarm: 
Alarm  is  within 
0.76  meters,  but 
line  perpendicular 
to  direction  of 
travel  does  not 
cross  halo 


Scoring  a  Typical  Detection  Mission 

IE  =  3  targets  encountered 
IR  =  8  reported  detections 
ID  =  2  scored  detections 
IAa=  1  additional  detection 
IFA=4false  alarms 
FAR  =  4/linear  kilometers  scanned 
Pd  =2/3  or  66.7% 


Figure  G-l.  Performance  detection  scoring  and  assessment  example, 
g.  Performance  detection  scoring  calculation  formulas  are  as  follows: 


PD  =  ED/EE 
Aa  =  Z(A  -  D) 
ZFA  =  ZR  -  ZA 
FAR  =  ZF  A/length 


Where 


E  =  an  encounter  of  a  target 
R  =  a  reported  alert 

A  =  a  reported  alert  within  the  required  accuracy 

D  =  a  single,  “scored”  accurate  alert  for  a  single  encounter 

Aa  =  number  of  additional  accurate  alerts 

FA  =  false  alarm;  a  reported  alert  outside  the  allowed  accuracy  of 
any  target 

FAR  =  false  alarm  rate;  the  sum  of  FAs  per  length  searched. 
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G.2  RECEIVER  OPERATING  CHARACTERISTIC  (ROC)  CURVES. 

a.  The  Pd  is  the  probability  of  saying  that  "1"  is  true  given  that  event  "1"  occurred.  The 
Pfa  is  the  probability  of  saying  that  "1"  is  true  given  that  the  "0"  event  occurred.  In  applications 
such  as  sonar  and  radar,  the  "1"  event  indicates  that  a  target  is  present,  and  the  "0"  event 
indicates  that  a  target  is  not  present. 

b.  A  detector’s  performance  is  measured  by  its  ability  to  achieve  a  certain  probability  of 
detection  and  probability  of  false  alarm  for  a  given  SNR.  Examining  a  detector's  ROC  curves 
provides  insight  into  its  performance.  We  can  use  the  rocsnr  function  to  calculate  and  plot  ROC 
curves. 


c.  One  feature  of  the  rocsnr  function  is  that  you  can  specify  a  vector  of  SNR  values  and 
rocsnr  calculates  the  ROC  curve  for  each  of  these  SNR  values.  Instead  of  individually 
calculating  Pd  and  Pfa  values  for  a  given  SNR,  we  can  view  the  results  in  a  plot  of  ROC  curves. 
The  rocsnr  function  plots  the  ROC  curves  by  default  if  no  output  arguments  are  specified. 
Calling  the  rocsnr  function  with  an  input  vector  of  four  SNR  values  and  no  output  arguments 
produces  a  plot  of  the  ROC  curves. 

d.  In  the  plot  (Figure  G-2)  we  can  select  the  data  cursor  button  in  the  toolbar  (or  in  the 
Tools  menu)  and  then  select  the  SNR  =  8  dB  curve  at  the  point  where  Pd  =  0.9  to  verily  that  Pfa 
is  approximately  0.01. 
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SNRvals  =  [24  8  9.4]; 
rocsnr(SNRvals); 


Nonfluctuating  Coherent  Receiver  Operating  Characteristic  (ROC)  Curves 


Figure  G-2.  ROC  curve. 


G.3  MULTIPLE  PULSE  DETECTION. 

a.  One  way  to  improve  a  detector's  performance  is  to  average  over  several  pulses.  This  is 
particularly  useful  in  cases  where  the  signal  of  interest  is  known  and  occurs  in  additive  complex 
white  noise.  Although  this  still  applies  to  both  linear  and  square-law  detectors,  the  result  for 
square-law  detectors  could  be  off  by  about  0.2  decibel  (dB).  Let's  continue  our  example  by 
assuming  an  SNR  of  8  dB  and  averaging  over  two  pulses. 
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b.  By  inspecting  the  plot  (Figure  G-3)  we  can  see  that  averaging  over  two  pulses  resulted 
in  a  higher  probability  of  detection  for  a  given  false  alarm  rate.  With  an  SNR  of  8  dB  and 
averaging  over  two  pulses,  you  can  constrain  the  probability  of  false  alarm  to  be  at  most  0.0001 
and  achieve  a  probability  of  detection  of  0.9.  Recall  that  for  a  single  pulse,  we  had  to  allow  the 
probability  of  false  alarm  to  be  as  much  as  1%  to  achieve  the  same  probability  of  detection. 


rocsnr(8,'NumPulses',2) 


Nonfluctuating  Coherent  Receiver  Operating  Characteristic  (ROC)  Curves 


Figure  G-3.  ROC  curve. 
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H.l.  The  data  detection  perfonnance  results  from  formal  testing  of  mature  system  design 
configurations  are  classified  as  specified  below. 

a.  Statements  regarding  quantitative  defeat  perfonnance  at  specific  (quantitative)  depths 
against  specific  models  of  U.S.  landmines  are  SECRET.  Statements  regarding  performance 
against  generic  U.S.  landmine  types  (e.g.,  simulators  or  surrogates)  or  generic  depths  (e.g., 
shallow  or  deep)  or  statements  in  which  the  system  is  not  identified  (unless  it  is  identified  in  the 
larger  context  of  a  document)  are  UNCLASSIFIED.  Examples  are  as  follows: 

(1)  SECRET:  Name  of  SUT  True  Name  defeats  XX%  of  specific  U.S.  landmine 
nomenclature  at  specific  depths. 

(2)  UNCLASSIFIED:  Name  of  SUT  defeats  XX%  of  specific  U.S.  landmine 
nomenclature  at  generic  depths. 

(3)  UNCLASSIFIED:  "  SUT-Y"  (coded  name)  defeats  XX%  of  specific  U.S. 
landmine  nomenclature  at  specific  depths. 

(4)  UNCLASSIFIED:  "  SUT-Y"  (coded  name)  defeats  XX%  of  specific  U.S. 
landmine  nomenclature  at  generic  depths. 

(5)  UNCLASSIFIED:  Name  of  SUT  defeats  specific  U.S.  landmine  nomenclature. 

(6)  UNCLASSIFIED:  Name  of  SUT  defeats  XX%  of  generic  U.S.  landmine  type  at 
specific  depths. 

(7)  UNCLASSIFIED:  Name  of  SUT  defeats  XX%  of  generic  U.S.  landmine  type  at 
generic  depths. 

b.  Statements  regarding  quantitative  defeat  perfonnance  at  specific  (quantitative)  depths 
against  specific  models  foreign  landmines  are  SECRET.  Statements  regarding  performance 
against  generic  foreign  landmine  types  or  generic  depths  (e.g.,  shallow  or  deep)  or  statements  in 
which  the  system  is  not  identified  (unless  it  is  identified  in  the  larger  context  of  a  document)  are 
UNCLASSIFIED.  Examples  are  as  follows: 

(1)  SECRET:  Name  of  SUT  defeats  XX%  of  specific  foreign  landmine  nomenclature 
at  specific  depths. 

(2)  UNCLASSIFIED:  Name  of  SUT  defeats  XX%  of  specific  foreign  landmine 
nomenclature  at  generic  depth. 
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(3)  UNCLASSIFIED:  "  SUT-Y"  (coded  name)  defeats  XX%  of  specific  foreign 
landmine  nomenclature  at  specific  depths. 

(4)  UNCLASSIFIED:  "  SUT-Y"  (coded  name)  defeats  XX%  of  specific  foreign 
landmine  nomenclature  at  generic  depths. 

(5)  UNCLASSIFIED:  NameofSUT  defeats  specific  foreign  landmine  nomenclature. 

(6)  UNCLASSIFIED:  Name  of  SUT  defeats  XX%  of  generic  foreign  landmine  type 
at  specific  depths. 

(7)  UNCLASSIFIED:  Name  of  SUT  defeats  XX%  of  generic  foreign  landmine  type 
at  generic  depths. 

c.  Statements  regarding  quantitative  defeat  perfonnance  at  specific  (quantitative)  depths 
against  specific  electromagnetic  hazard  (EH)  other  than  landmines  (non-landmine  EH),  which 
are  IEDs  (high  explosive  component,  excluding  the  trigger  mechanism,  see  d  below)  are 
SECRET.  Statements  regarding  performance  against  generic  non-landmine  EH  or  generic 
depths  (e.g.,  shallow  or  deep)  or  statements  in  which  the  system  is  not  identified  (unless  it  is 
identified  in  the  larger  context  of  a  document)  are  UNCLASSIFIED.  Examples  are  as  follows: 

(1)  SECRET:  Name  of  SUT  defeats  XX%  of  specific  non-landmine  EH  at  specific 

depths. 

(2)  UNCLASSIFIED:  Name  of  SUT  defeats  XX%  of  specific  non-landmine  EH 
nomenclature  at  generic  depth. 

(3)  UNCLASSIFIED:  "SUT  Y"  (coded  name)  defeats  XX%  of  non-landmine  EH 
nomenclature  at  specific  depths. 

(4)  UNCLASSIFIED:  "SUT  Y"  (coded  name)  defeats  XX%  of  specific  non-landmine 
EH  nomenclature  at  generic  depths. 

(5)  UNCLASSIFIED:  Name  of  SUT  defeats  specific  non-landmine  EH 
nomenclature. 

(6)  UNCLASSIFIED:  Name  of  SUT  defeats  XX%  of  generic  non-landmine  EH  type 
at  specific  depths. 

(7)  UNCLASSIFIED:  Name  of  SUT  defeats  XX%  of  generic  non-landmine  EH  type 
at  generic  depths. 
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d.  Statements  regarding  quantitative  defeat  perfonnance  against  specific  trigger 
mechanisms  used  to  actuate  IEDs,  regardless  of  depth,  are  SECRET.  Statements  regarding 
performance  against  trigger  mechanisms  in  general,  or  statements  in  which  the  system  is  not 
identified  (unless  it  is  identified  in  the  larger  context  of  a  document),  are  UNCLASSIFIED. 
Examples  are  as  follows: 

(1)  SECRET :  Name  of  SUT  defeats  XX%  of  (specific  trigger  mechanism 
description). 

(2)  UNCLASSIFIED:  "SUT  Y"  (coded  name)  defeats  XX%  of  (specific  trigger 
mechanism  description). 

(3)  UNCLASSIFIED:  Name  of  SUT  defeats  (specific  trigger  mechanism 
description). 

(4)  UNCLASSIFIED:  Name  of  SUT  defeats  XX%  of  trigger  mechanism. 

H.2.  The  release  of  all  test  data  should  be  in  accordance  with  the  project  Security  Classification 
Guide  and  ATEC  Reg  73-1 1S. 
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Aa 

additional  accurate  detection 

ACU 

Army  Combat  Uniform 

AEC 

U.S.  Army  Evaluation  Center 

ANSI 

American  National  Standards  Institute 

AP-LM 

antipersonnel  low  metallic 

AP-M 

antipersonnel  metallic 

AR 

Advance  Rate 

AT-LM 

anti-tank  low  metallic 

AT-M 

anti-tank  metallic 

ATEC 

U.S.  Army  Test  and  Evaluation  Command 

ATIRS 

Army  Test  Incident  Reporting  System 

BIT 

Built-In  Test 

BITE 

Built-In  Test  Equipment 

C 

Celsius 

cm 

centimeter 

CM 

counter  measure 

ConOps 

Concept  of  Operations 

CREW 

counter  radio  electronic  warfare 

CTTR 

Countermine  Testing  and  Training  Range 

CW 

continuous  wave 

DA  PAM 

Department  of  the  Army  Pamphlet 

dB 

decibel 

D-GPS 

differential  global  positioning  system 

E 

East 

E3 

electromagnetic  enviromnental  effects 

EM 

electromagnetic 

EMC 

electromagnetic  compatibility 

EH 

electromagnetic  hazard 

EMI 

electromagnetic  interference 

EMR 

electromagnetic  radiation 

F 

Fahrenheit 

FA 

False  Alarm 

FAR 

False  Alarm  Rate 

FOUO 

For  Official  Use  Only 

FSR 

Field  Service  Representative 

GPR 

ground  penetrating  radar 

GPS 

global  positioning  system 
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HEMP 

High  altitude  electromagnetic  pulse 

HERF 

Hazards  of  Electromagnetic  Radiation  to 

HERO 

Hazards  of  Electromagnetic  Radiation  to 

HERP 

Hazards  of  Electromagnetic  Radiation  to 

HESD 

Helicopter  Electrostatic  Discharge 

IAW 

in  accordance  with 

ID 

identification 

IED 

improvised  explosive  device 

IRIG 

Intra-Range  Instrumentation  Group 

ITOP 

International  Test  Operations  Procedure 

kg 

kilogram 

LED 

light-emitting  diode 

LMPP 

low  metal  pressure  plate 

LS 

logistic  support 

m 

meter 

MANPRINT 

Manpower  and  Personnel  Integration 

MIL-STD 

Military  Standard 

mm 

millimeter 

MMC 

metal  main  charge 

MOPP 

mission  oriented  protective  posture 

MRAP 

mine-resistant  ambush  protected 

MSDS 

material  safety  data  sheet 

N 

North 

NATO 

North  Atlantic  Treaty  Organization 

NBC 

nuclear,  biological,  chemical 

NMMC 

no  metal  main  charge 

NMPP 

no  metal  pressure  plate 

OM 

Operator  Manual 

OPSEC 

Operations  Security 

Pd 

probability  of  detection 

Pd 

performance  of  detection 

PD 

percent  detection 

PEL 

personnel  exposure  limits 

PESD 

personnel  electrostatic  discharge 

Pfa 

probability  of  false  alarm 

Fuel 

Ordnance 

Personnel 
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PHA 

Preliminary  Hazard  Assessment 

PII 

personal  identification  infonnation 

PM 

Program  Manager 

PMCS 

preventive  maintenance  checks  and  services 

PPIED 

pressure-plate  improvised  explosive  device 

QA 

quality  assurance 

QC 

quality  control 

RAM 

reliability,  availability,  and  maintainability 

RCIED 

remote  controlled  improvised  explosive  device 

RCRIT 

critical  radius 

REC 

Record  of  Environmental  Consideration 

Reg 

regulation 

RF 

radio  frequency 

ROA 

rate  of  advance 

ROC 

Receiver  Operating  Characteristic 

S 

South 

SAR 

Safety  Assessment  Report 

ScR 

scan  rate 

SME 

subject  matter  expert 

SMO 

Spectrum  Management  Office 

SNR 

signal-to-noise  ration 

soc 

state  of  charge 

SOP 

Standard  Operation  Procedure 

SR 

Safety  Release 

SSP 

system  support  package 

STANAG 

Standardization  Agreement 

SUT 

system  under  test 

SwR 

sweep  rate 

TEMP 

Test  and  Evaluation  Master  Plan 

TIR 

test  incident  report 

TM 

Technical  Manual 

TOP 

Test  Operations  Procedure 

TSARC 

Test  Schedule  and  Review  Committee 

uses 

Unified  Soil  Classification  System 

UTM 

Universal  Transverse  Mercator 

W 

West 

WGS 

World  Geodetic  System 
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CSTE-TM  8  January  20 1 5 


MEMORANDUM  FOR 

Commanders.  All  Test  Centers 
Technical  Directors.  All  Test  Centers 
Directors,  U  S  Army  Evaluation  Center 
Commander.  U  S  Army  Operational  Test  Command 

SUBJECT:  Test  Operations  Procedure  (TOP)  04-2-090A,  Testing  of  Hand  Held  Mine 
Detection  Systems.  Approved  for  Publication 


1.  TOP  04-2-090A.  Testing  of  Hand  Held  Mine  Detection  Systems,  has  been  reviewed 
by  the  U  S.  Army  Test  and  Evaluation  Command  (ATEC)  Test  Centers,  the  U  S.  Army 
Operational  Test  Command,  and  the  U  S.  Army  Evaluation  Center.  All  comments 
received  during  the  formal  coordination  period  have  been  adjudicated  by  the  prepanng 
agency  The  scope  of  the  document  is  as  follows: 

The  purpose  of  this  TOP  is  to  provide  the  tester  standardized  testing  methodologies 
and  procedures  to  assess  handheld  detection  systems  in  order  to  test  the  overall  safety, 
performance,  and  reliability  of  the  system.  It  describes  activities  necessary  to  ensure 
safety  is  designed  into  the  system  under  test,  and  to  verify  detection  performance  meets 
system  requirements  (to  detect  landmines,  improvised  explosive  devices  and  their 
components,  as  well  as  ammunition  cache  and  unexploded  ordnance). 

2.  This  document  is  approved  for  publication  and  has  been  posted  to  the  Reference 
Library  of  the  ATEC  Vision  Digital  Library  System  (VDLS).  The  VDLS  website  can  be 
accessed  at  https://vdls.atc.amny  mil/. 

3.  Comments,  suggestions,  or  questions  on  this  document  should  be  addressed  to  U  S. 
Army  Test  and  Evaluation  Command  (CSTE-TM),  2202  Aberdeen  Boulevard-Third  Floor, 
Aberdeen  Proving  Ground,  MD  21005-5001;  or  e-mailed  to  usarmy.apg  atec  mbx  atec- 
standards@mail.mil. 


FONTAINE.RAYMO  ^  7s 
ND  G1228612770  ZZ-ZZ. _ _ 

tarn  »it»n  n  ■  mm 

RAYMOND  G.  FONTAINE 

Associate  Director,  Test  Management  Directorate  (G9) 


FOR 


MICHAEL  J.  ZWIEBEL 

Director,  Test  Management  Directorate  (G9) 


K-l 


TOP  04-2-090A 
8  January  2015 


(This  page  is  intentionally  blank.) 


K-2 


TOP  04-2-090A 
8  January  2015 


Forward  comments,  recommended  changes,  or  any  pertinent  data  which  may  be  of  use  in 
improving  this  publication  to  the  following  address:  Range  Infrastructure  Division  (CSTE-TM), 
U.S.  Army  Test  and  Evaluation  Command,  2202  Aberdeen  Boulevard,  Aberdeen  Proving 
Ground,  Maryland  21005-5001.  Technical  information  may  be  obtained  from  the  preparing 
activity:  TEDT-YPY-MWM  (Munitions  and  Weapons  Division),  U.S.  Army  Yuma  Proving 
Ground,  Yuma,  Arizona  85365.  Additional  copies  can  be  requested  through  the  following 
website:  http://www.atec.armv.mil/publications/topsindex.aspx,  or  through  the  Defense 
Technical  Information  Center,  8725  John  J.  Kingman  Rd.,  STE  0944,  Fort  Belvoir,  VA  22060- 
6218.  This  document  is  identified  by  the  accession  number  (AD  No.)  printed  on  the  first  page. 


